Tate was born near Winchester, Kentucky, to John Orley Tate, a businessman, and Eleanor Parke Custis Varnell. In 1916 and 1917 Tate studied the violin at the Cincinnati Conservatory of Music.

He began attending Vanderbilt University in 1918, where he met fellow poet Robert Penn Warren. Warren and Tate were invited to join an informal literary group of young Southern poets under the leadership of John Crowe Ransom; the group were known as the Fugitives. Tate contributed to the group's magazine The Fugitive. The aim of the group, according to the critic J. A. Bryant, was "to demonstrate that a group of southerners could produce important work in the medium [of poetry], devoid of sentimentality and carefully crafted," and they wrote in the formalist tradition that valued the skillful use of meter and rhyme.

In 1924, Tate moved to New York City where he met poet Hart Crane, with whom he had been exchanging correspondence for some time. Over a four-year period, he worked freelance for The Nation, contributed to the Hound & Horn, Poetry magazine, and others. To make ends meet, he worked as a janitor. (Some years later, he would also contribute articles to the conservative National Review.)

Tate married Gordon in New York in May 1925. Their daughter Nancy was born in September. In 1928, along with others New York City friends, he went to Europe. In London, he visited with T. S. Eliot, whose poetry and criticism he greatly admired, and he also visited Paris.

Just before leaving for Europe in 1928, Tate described himself to John Gould Fletcher as "an enforced atheist". He later told Fletcher, "I am an atheist, but a religious one — which means that there is no organization for my religion." He regarded secular attempts to develop a system of thought for the modern world as misguided. "Only God," he insisted, "can give the affair a genuine purpose." In his essay "The Fallacy of Humanism" (1929), he criticized the New Humanists for creating a value system without investing it with any identifiable source of authority. "Religion is the only technique for the validation of values," he wrote. Although he was attracted to Roman Catholicism, he deferred converting. Louis D. Rubin, Jr. observes that Tate may have waited "because he realized that for him at this time it would be only a strategy, an intellectual act".

In 1929, Tate published a second biography Jefferson Davis: His Rise and Fall.

From the 1930s until as late as the 1960s, Tate held prejudices against both blacks and Jews. He expressed views against interracial marriage and miscegenation and refused to associate with Black Writers (like the aforementioned Langston Hughes). Up until the 1960s, Tate also believed in white supremacy.

In 1933, Tate wrote a letter for Hound & Horn explaining his views on interracial sex. "The negro race is an inferior race....miscegenation due to a white woman and a negro man" threatened the white family. "Our purpose..is to keep the negro blood from passing into the white race."

According to the poetry Editor of The New Criterion, David Yezzi, Tate held the conventional social views of a white Southerner in 1934: an "inherited racism, a Southern legacy rooted in place and time that Tate later renounced." Tate was born of a Scotch-Irish lumber manager whose Business failures required moving several times per year, Tate said of his upbringing ""we might as well have been living, and I been born, in a tavern at a crossroads." However, his views on race were not passively incorporated; Thomas Underwood documents Tate's pursuit of racist ideology: "Tate also drew ideas from nineteenth-century proslavery theorists such as Thomas Roderick Dew, a professor at The College of william and Mary, and william Harper, of the University of South Carolina — "We must revive these men, he said."

During this time, Tate also became the de facto associate Editor of The American Review, which was published and edited by Seward Collins. Tate believed The American Review could popularize the work of the Southern Agrarians. He objected to Collins's open support of Benito Mussolini and Adolf Hitler, and condemned Fascism in an article in The New Republic in 1936. Much of Tate's major volumes of poetry were published in the 1930s, and the scholar David Havird describes this publication history in poetry as follows:

In 1938 Tate published his only novel, The Fathers, which drew upon knowledge of his mother's ancestral home and family in Fairfax County, Virginia.

Tate was a poet-in-residence at Princeton University until 1942. He founded the Creative Writing program at Princeton, and mentored Richard Blackmur, John Berryman, and others. In 1942, Tate assisted Novelist and friend Andrew Lytle in transforming The Sewanee Review, America's oldest literary quarterly, from a modest journal into one of the most prestigious in the nation. Tate and Lytle had attended Vanderbilt together prior to collaborating at The University of the South.

Tate and Gordon were divorced in 1945 and remarried in 1946. Though devoted to one another for life, they could not get along and later divorced again.

In 1950, Tate converted to Roman Catholicism. He also married the poet Isabella Gardner in the early 1950s.

While teaching at the University of Minnesota in Minneapolis, he met Helen Heinz, a nun enrolled in one of his courses and began an affair with her. Tate divorced Gardner and married Heinz in 1966. They moved to Sewanee, Tennessee. In 1967, Tate became the father of twin sons. The youngest died at eleven months from an accident. A third son was born in 1969. Tate died in Nashville, Tennessee ten years later. His papers are collected at the Firestone Library at Princeton University.