Liebling was born into a well-off family on the Upper East Side of Manhattan, where his father worked in New York's fur industry. His mother, Anna Adelson Slone, was from San Francisco. After early schooling in New York, Liebling was admitted to Dartmouth College in the fall of 1920. His primary activity during his undergraduate career was as a contributor to the Jack-O-Lantern, Dartmouth's nationally known humor magazine. He left Dartmouth without graduating, later claiming he was "thrown out for missing compulsory chapel attendance". He then enrolled in the School of Journalism at Columbia University.

Thus in summer 1926, Liebling sailed to Europe where he studied French medieval literature at the Sorbonne in Paris. By his own admission his devotion to his studies was purely nominal, he seeing the year as a chance to absorb French life and appreciate French food. Although he stayed for little more than a year, this interval inspired a lifelong love for France and the French, later renewed in his war reporting. He returned to Providence in autumn 1927 to write for the Journal. He then moved to New York, where he proceeded to campaign for a job on Joseph Pulitzer's New York World, which carried the work of James M. Cain and Walter Lippmann and was known at the time as "the writer's paper." In order to attract the attention of the city Editor, James W. Barrett, Liebling hired an out-of-work Norwegian seaman to walk for three days outside the Pulitzer Building, on Park Row, wearing sandwich boards that read Hire Joe Liebling. It turned out that Barrett habitually used a different entrance on another street, and never saw the sign. He wrote for the World (1930–31) and the World-Telegram (1931–35).

He married Ann Beatrice McGinn, a former movie theater ticket taker he had met while she was working in Providence, Rhode Island, on July 28, 1934. McGinn suffered from either manic depression or schizophrenia, which caused her to have hallucinations and go into fugue states. Her illness required many lengthy and expensive hospital stays and when she was out of the hospital, she was often heavily sedated. Both Liebling and McGinn committed infidelities during their marriage.

Liebling joined The New Yorker in 1935. His best pieces from the late thirties are collected in Back Where I Came From (1938) and The Telephone Booth Indian (1942).

During World War II, Liebling was active as a war correspondent, filing many stories from Africa, England, and France. His war began when he flew to Europe in October 1939 to cover its early battles, lived in Paris until June 10, 1940, and then returned to the United States until July 1941, when he flew to Britain. He sailed to Algeria in November 1942 to cover the fighting on the Tunisian front (January to May 1943). His articles from these days are collected in The Road Back to Paris (1944). He participated in the Normandy landings on D Day, and he wrote a memorable piece concerning his experiences under fire aboard a U.S. Coast Guard-manned landing craft off Omaha Beach. He afterwards spent two months in Normandy and Brittany, and was with the Allied forces when they entered Paris. He wrote afterwards: "For the first time in my life and probably the last, I have lived for a week in a great city where everybody was happy." Liebling was awarded the Cross of the Légion d'honneur by the French government for his war reporting.

In 1947 he published The Wayward Pressman, a collection of his writings from The New Yorker and other publications.

In 1946 he and his wife separated. They divorced on August 30, 1949, in Reno, Nevada. Two days later he married Lucille Spectorsky, the ex-wife of Auguste Comte Spectorsky, in Virginia City, Nevada. Spectorsky was described by Liebling's friend and New Yorker Editor Gardner Botsford as "a big blonde from rural Kentucky, amiable if dumb." Liebling and Spectorsky divorced in 1959 and he married author Jean Stafford that same year.

In 1961, Liebling published The Earl of Louisiana, originally published as a series of articles in The New Yorker in which he covered the trials and tribulations of the governor of Louisiana, Earl K. Long, the younger brother of the Louisiana Politician Huey Long.

His writing was often memorable, as was his eating, and he nicely combined the two passions in Between Meals (1962), of which the following extract gives a taste:

On December 19, 1963, Liebling was hospitalized for Bronchopneumonia. He died on December 28 at Mount Sinai Hospital, and was buried in the Green River Cemetery, East Hampton, New York.