(Photo: Mary Ellen Mark)

It isn’t until our third interview that I notice Gay Talese has been sitting underneath a painting of a naked woman with a rainbow coming out of her vagina. We are in the otherwise staid living room of his townhouse in the East Sixties, the home where he and his wife, Nan, the publisher of Nan A. Talese/Doubleday books, have lived for half a century. In fact, in June they will celebrate their 50th wedding anniversary, an impressive milestone for any couple, but perhaps few more so than this one. This legendary literary marriage—in all of its baroque complexity—has taken place entirely under this roof. It is here where they began their life together as a couple in their mid-twenties, when the five-story brownstone was like a tenement and Gay lived in 3F, a studio they both still refer to as his “bachelor pad”; this is where they raised their two daughters, Pamela and Catherine, and slowly took over every apartment in the building before finally buying it in 1973; this is where they have held innumerable book parties for Nan’s celebrated authors; and this is where Gay has done much of the writing—the historic Esquire pieces, the best sellers with biblical titles—that brought him fame, fortune, and no small amount of personal agony.

What’s absurd about the fact that I missed the rainbow vagina painting is that the subject of our conversation is, in so many words, sex. This month, Ecco re-published Thy Neighbor’s Wife, with a foreword by Katie Roiphe (along with a new edition of Honor Thy Father, foreword by Pete Hamill). The book, originally published in 1980, is about the sexual revolution, which Talese believed would be the most important cultural shift in decades, and which he spent most of the seventies intimately researching. It’s the research itself—particularly Talese’s tendency to take the participant-observer concept to the extreme—that turned out to be the unintended legacy of the project. “If you want to write about orgies,” says Talese, who at 77 is still slim and handsome, “you’re not going to be in the press box with your little press badge keeping your distance. You have to have a kind of affair with your sources. You have to hang out! I wanted to write about sexuality and the changing definition of morality. Maybe if I had put that in a subhead on the cover I might have gotten a better hearing.”

Instead, the book garnered him the worst reviews of his career (while also making him millions—the $2.5 million he was paid for film rights remained a record until 1991). Part of the reason critics were so appalled was because while Talese was gallivanting promiscuously and publicly, he was married, and not just to some anonymous wife but to Nan Talese, an important book editor at Random House whom many of the critics knew personally, professionally, or socially. Gay writes in his new afterword that in the immediate aftermath of the book’s publication, Nan accompanied him on talk shows “to explain that our marital love had remained unthreatened while I conducted research in New York massage parlors and a hedonistic nudist colony in Los Angeles”—which is only half true. As he has been telling me in exquisite detail, their marriage was pushed to the breaking point during this time. And so was his career.

Fair or not, it is a commonly held opinion in publishing circles that Talese’s career can be pretty much divided into pre– and post–Thy Neighbor’s Wife—that the writer and his gift never fully recovered from the shock waves. By 1980, he rarely wrote for magazines anymore, and it was a full twelve years before he published another book, Unto the Sons, which traced his Italian family’s immigrant history. Rumor had it that he was blocked, though Talese insists that wasn’t the case. He did, however, seek professional help. “One guy I saw for years was a Freudian,” says Talese. “The shrink was a very nice guy. He liked the book. But he said, ‘What you did was commit literary suicide.’ ”

It’s difficult to recover from such a leap into ignominy, and Talese has spent the decades since collecting material—photos, letters, anecdotes—about himself and Nan, as if looking for clues to his character’s motivations. In recent years, the collecting has grown more purposeful, and Talese has settled on an ambitious new project, the subject of which will take him right back to the scene of the crime, to the spot where everything went off the rails: a book about his marriage.

I ask him if the marriage book is an attempt to analyze what happened all those years ago.

“Yeah,” he says. “I have never dealt with it.”