Kluge was born to a Presbyterian family in Chemnitz, Germany, and emigrated to the United States in 1922. He earned his B.A. degree in economics from Columbia University in 1937. Prior to attending Columbia University, Kluge went to Wayne State University for two years. He was of Scots-Irish, English, and German heritage.

Kluge's major move into media was by purchasing stock in the Metropolitan Broadcasting Corporation in the mid-1950s. The Metropolitan Broadcasting Corporation was the successor of the DuMont Television Network, which was spun off from DuMont Laboratories after the television network ceased operations in 1956. Metropolitan Broadcasting consisted of two stations, WABD in New York City and WTTG in Washington, D.C., both former DuMont outlets now operating as independent stations. Kluge joined the company as its board chairman and largest stockholder in 1958, acquiring the bulk of his shares from founder Allen B. DuMont for about USD $6,000,000.

After gaining control in 1959, Kluge began the company's expansion further into broadcasting, with holdings in television and radio. In the early 1960s, Kluge bought an outdoor advertising firm, and in 1961 the company's name was changed to Metromedia to reflect the diversity of its interests.

In 1986, Kluge sold the Metromedia television stations to the 20th Century Fox film studio, which is now controlled by 21st Century Fox, for a reported USD $4 billion. Those stations would later form the core of what would become the Fox television network. The following year, Forbes placed Kluge at the top of its list as the richest man in America.

In December 1997, John Kluge gifted the bulk his world-class collection of Aboriginal Australian art to the University of Virginia. The gift led the University of Virginia to found the Kluge-Ruhe Aboriginal Art Museum, the only museum in the United States dedicated to the research and presentation of Aboriginal Australian art and culture.

In 2001, Kluge donated his 7,378-acre (29.86 km) estate in Albemarle County, Virginia to the University of Virginia. The estate, valued in excess of $45 million, was the largest gift in the University's history. UVa has been holding classes and seminars in the various buildings and on the grounds of Morven Farm in an effort to incorporate the land grant into their various course offerings. Many developments are in the works to improve the farm and make it part of the landscape of Charlottesville.

Kluge gave over $510 million to Columbia University. Acknowledging the scholarship funds that enabled him to attend, Kluge gave more than $110 million to Columbia University between 1987 and 1993, primarily to endow financial aid for undergraduates from underprivileged backgrounds. His donations also help many of these students pursue Ph.D.s after they graduate by financing their doctoral studies. On April 11, 2007, Columbia University's President Lee C. Bollinger announced a $400 million pledge from Kluge, which the university was to receive upon the donor's death. The donation marks the fourth largest gift to an institution of higher learning in America, all designated for financial aid. This marks the largest pledge ever devoted exclusively to student aid to any single institution of higher education in the United States.

Following the Fox disposal, Kluge's activities had been carried out through a private venture named Metromedia Company in which he was a partner with Stuart Subotnick. Metromedia's more recent activities have included Eastern European, Commonwealth of Independent States and China telecom/cable/radio ventures through Metromedia International Group and the ill-fated US Telecom backbone operation Metromedia Fiber Network. In July 2008, the Metromedia Restaurant Group, part of the Metromedia Company, closed over 300 company-owned Bennigan's and Steak and Ale restaurants. Kluge and partner Stuart Subotnick were also the original team operators of the New York/New Jersey MetroStars Major League Soccer franchise.

Kluge was married four times. His first wife was Theodora Thomson Townsend, his second was Yolanda Galardo Zucco, the third was Patricia Maureen Rose, and his fourth was Maria Tussi Kuttner. Patricia Kluge has since filed for bankruptcy after taking on too much leverage during the recession. Kluge had three children, Joseph (whom he adopted), Samantha (with Zucco) and John Jr (adopted, with Rose). He also had three step children who remained a part of his life until his death: Diane, Jeanette Brophy and Peter Lockwood Townsend. He had homes in New Rochelle, New York, Virginia and Palm Beach, Florida with his fourth wife, Maria Tussi Kluge, at the time of his death in 2010.