Anatole Broyard was born in 1920 in New Orleans, Louisiana, into a mixed-race Louisiana Creole family, the son of Paul Anatole Broyard, a carpenter and construction worker, and his wife, Edna Miller, neither of whom had finished elementary school. Broyard was descended from ancestors who were established as free people of color before the Civil War. The first Broyard recorded in Louisiana was a French colonist in the mid-eighteenth century. Broyard was the second of three children; he and his sister Lorraine, two years older, were light-skinned with European features. Their younger sister, Shirley, who eventually married Franklin Williams, an attorney and civil rights leader, had darker skin and African features.

Novelist william Gaddis, who likewise knew Broyard in the late 1940s, modeled a character named "Max" on Broyard in his first novel, The Recognitions (1955).

He often was said to be working on a novel, but never published one. After the 1950s, Broyard taught creative writing at The New School, New York University, and Columbia University, in addition to his regular book reviewing. For nearly fifteen years, Broyard wrote daily book reviews for the New York Times. The Editor John Leonard was quoted as saying, "A good book review is an act of seduction, and when he [Broyard] did it there was no one better."

Broyard did not identify with or champion black political causes. Because of his artistic ambition, in some circumstances he never acknowledged that he was partially-black. On the other hand, Margaret Harrell has written that she and other acquaintances were casually told that he was a Writer and black before meeting him, and not in the sense of having to keep it secret. That he was partially-black was well known in the Greenwich Village literary and art community from the early 1960s.

In 1961, at the age of 40, Broyard married again, to Alexandra (Sandy) Nelson, a modern Dancer and younger woman of Norwegian-American ancestry. He was indirect about his family, but she learned about him. They had two children: son Todd, born in 1964, and daughter Bliss, born in 1966. The Broyards raised their children as white in suburban Connecticut. When they had grown to young adults, Sandy urged Broyard to tell them about his family (and theirs), but he never did.

In the late 1970s, Broyard started publishing brief personal essays in the Times, which many people considered among his best work. These were collected in Men, Women and Anti-Climaxes, published in 1980. In 1984 Broyard was given a column in the Book Review, for which he also worked as an Editor. He was among those considered "gatekeepers" in the New York literary world, whose positive opinions were critical to a writer's success.

Broyard used the GI Bill to study at the New School for Social Research in Manhattan. He settled in Greenwich Village, where he became part of its bohemian artistic and literary life. With money saved during the war, Broyard owned a bookstore for a time. As he recounted in a 1979 column:

Sandy told their children of their father's secret before his death. Broyard died in October 1990 of prostate cancer, which had been diagnosed in 1989. His first wife and child were not mentioned in his New York Times obituary.

In 1996, six years after Broyard's death, Henry Louis Gates criticized the Writer, in a profile entitled "White Like Me" in The New Yorker, for concealing his African-American ancestry. Gates expanded his essay in "The Passing of Anatole Broyard", a piece published the next year in his Thirteen Ways of Looking at a Black Man (1997). Gates felt that Broyard had deceived friends and family by "passing" as white, but also understood his literary ambition. He wrote,

Given Broyard's stature in the literary world and discussions about his life after his death, numerous literary critics, such as Michiko Kakutani, Janet Maslin, Lorrie Moore, Charles Taylor, Touré, and Brent Staples, have made comparisons between the character Coleman Silk in Philip Roth's The Human Stain (2000) and Broyard.

As Writer and Editor Brent Staples wrote in 2003, "Anatole Broyard wanted to be a Writer -- and not just a 'Negro writer' consigned to the back of the literary bus." The Historian Henry Louis Gates, Jr. wrote: "In his terms, he did not want to write about black love, black passion, black suffering, black joy; he wanted to write about love and passion and suffering and joy."

In 2007, Broyard's daughter Bliss published a memoir, One Drop: My Father's Hidden Life: A Story of Race and Family Secrets. (The title related to the "one-drop rule". Adopted into law in most southern states in the early twentieth century, it divided society into two groups, whites and blacks, classifying all persons with any known black ancestry as black.) Her book explored her psychological and physical journeys as she met members of her father's extended family in New York, New Orleans, and on the West Coast, and her developing ideas about her own identity and life.

(In his mini-series, African American Lives 2 (2008), Gates learned by DNA analysis and genealogical studies that he was more than half white by ancestry.)