James Frey: Five Years Later, Part 1

Though James has received some support and positive feedback from readers, he says that the negative attention was overwhelming. "I was able to go places and not feel ostracized, but for [weeks], it was weird and hard," he says. "I would walk out the door and there would be reporters there. There would be cameramen there."

James says he was also sued several times after his memoir was revealed to be both fact and fiction. "On more than one occasion, I was walking down the street and somebody walked up and served me with legal papers," he says.

He continues, "We got sued for damages. ... I got sued for a lot of stuff by a lot of people. I had to hire attorneys, and ultimately, we settled all of it. We gave every reader who had purchased the book, up until that point, the opportunity to get their money back. It was surreal, and I was having a really hard time."

With the public scrutiny and frequent news reports on the memoir controversy, James decided to leave the country and live in France with his wife and daughter. "I love France, and it was great to just push my daughter's stroller down the street and just be some anonymous dad," he says.