Hugh Hefner built an empire and created an entire lifestyle based on the simple premise that people are always fascinated by what is considered taboo.

Hefner, who founded Playboy in 1953 in part with a $1,000 loan from his mother and went on to become one of the most successful (and envied) entrepreneurs of the 20th century, died on Wednesday from natural cause at his home, the Playboy Mansion, surrounded by loved ones, according to Playboy Enterprises, Inc. He was 91 years old.

“I know there was a tremendous amount of curiosity about me and my lifestyle, and that is not altogether an accident,” Hefner told ET in 1982. “But I also think that has a major part to do with what Playboy’s all about. I think the two great fascinations and guilts in America are sex and wealth, and combining the two in Playboy — and then with my own lifestyle so intimately and related to the magazine — I think it’s very natural that people will feel a great fascination about that and about me.”

The image of Hefner as a silk pyjama-wearing man of leisure, complete with Playmates by his side, belies the fact that his business acumen turned Playboy into a publishing and branding behemoth in the 1960s and ’70s, the magazine’s heyday. At its peak, Playboy was selling more than 7 million copies a month, and Hefner had spun off the ubiquitous bunny ears into nightclubs and hotels.

MORE: Soul Singer Charles Bradley Dies at 68 After Cancer Battle

The first issue of Playboy (which would have been called Stag Party had Hefner been able to secure the name) featured Marilyn Monroe on the cover and was an instant success. From there, the magazine found a comfortable niche mixing centerfolds of nubile young women with in-depth interviews and pointed commentary from the newsmakers of the day — all part of the “Playboy Philosophy” Hefner detailed in a series of editorials over several years.

Hefner enjoyed being at the centre of it all, and certainly capitalized on the perks of being America’s most famous bon vivant. After he shifted his base of operations from Chicago to Los Angeles in 1975, his new home, the Playboy Mansion, became a worldwide symbol of decadence, and Hefner did little to dissuade that reputation, holding wild parties that mixed Hollywood celebrities with Hef’s bunnies. He squired a number of his centrefolds and dated several actresses, his sex life becoming a running joke for late-night hosts.

“How could I possibly know?” he remarked to Esquire when asked about the number of women he had slept with. “Over a thousand, I’m sure. There were chunks of my life when I was married, and when I was married I never cheated. But I made up for it when I wasn’t married. You have to keep your hand in.”

The 1980s were not as kind to Hefner or Playboy. Sales of the magazine dropped, former Playmate Dorothy Stratten was shot to death by her estranged husband, Playboy lost its casino licenses in London and Atlantic City and in 1985, Hefner suffered a minor stroke. In 1988, he turned over control of Playboy Enterprises to his daughter Christie, making her CEO and marking the end of an era for Hefner.

But Hefner never ceded the spotlight. In 1989, he married for the second time, to former Playmate Kimberley Conrad, with whom he had two children. He separated from Conrad in 1998 and seven years later found a new life on TV as the star of the reality program “The Girls Next Door”, which focused on his relationships with his girlfriends, making stars out of Holly Madison and Kendra Wilkinson.

“You’re talking to a human being here who is enjoying life and enjoys his work very much, but also enjoys the other parts of his life,” Hefner told ET. “It would be very foolish to have expounded a philosophy of working hard and playing hard and then myself not paid attention to what I was writing about.”

Hefner is survived by his wife Crystal, four grown children, Christie, who served as CEO of Playboy Enterprise for more than 20 years, David, Marston and Cooper, who currently serves as Chief Creative Officer at the company.

Information regarding memorial services will be given at a later date.

More From ET