Schrieffer was born in Oak Park, Illinois, and his family moved in 1940 to Manhasset, New York, and then in 1947 to Eustis, Florida, where his father a former pharmaceutical salesman began a career in the citrus industry. In his Florida days, Schrieffer enjoyed playing with homemade rockets and ham radio, a hobby that sparked an interest in electrical engineering.

After graduating from Eustis High School in 1949, Schrieffer was admitted to the Massachusetts Institute of Technology, where for two years he majored in electrical engineering before switching to physics in his junior year. He completed a bachelor's thesis on multiplets in heavy atoms under the direction of John C. Slater in 1953. Pursuing an interest in solid-state physics, Schrieffer began graduate studies at the University of Illinois at Urbana-Champaign, where he was hired immediately as a research assistant to John Bardeen. After working out a theoretical Problem of electrical conduction on semiconductor surfaces, Schrieffer spent a year in the laboratory, applying the theory to several surface problems. In his third year of graduate studies, he joined Bardeen and Leon Cooper in developing the theory of Superconductivity.

After completing his doctoral dissertation on the theory of Superconductivity, Schrieffer spent the 1957–58 academic year as a National Science Foundation fellow at the University of Birmingham in England and at the Niels Bohr Institute in Copenhagen, where he continued research into Superconductivity. Following a year as assistant professor at the University of Chicago, he returned to the University of Illinois in 1959 as a faculty member. In 1960, he went back to the Bohr Institute for a summer visit, during which he became engaged to Anne Grete Thomsen whom he married at Christmas of that year. Two years later, Schrieffer joined the faculty of the University of Pennsylvania in Philadelphia, and, in 1964, Schrieffer published his book on the BCS theory, Theory of Superconductivity. He holds honorary degrees from the Technical University of Munich and the University of Geneva. In 1993 Schrieffer along with Leon Cooper were awarded the Comstock Prize in Physics from the National Academy of Sciences.

In 1972, Schrieffer along with John Bardeen and Leon Cooper won the 1972 Nobel Prize in Physics for developing the BCS theory. In 1980, Schrieffer became a professor at the University of California, Santa Barbara, and rose to chancellor professor in 1984, serving as Director of the university’s Kavli Institute for Theoretical Physics. In 1992, Florida State University appointed Schrieffer as a university eminent scholar professor and chief scientist of the National High Magnetic Field Laboratory, where he continued to pursue one of the great goals in physics: room temperature Superconductivity.

Schrieffer was sentenced to two years in prison November 6, 2005 for vehicular manslaughter killing one, and injuring seven people. At the time of the accident his drivers license was under suspension and he is said to have fallen asleep at the wheel. The accident occurred in Orcutt, California on September 24, 2004, and Schrieffer was incarcerated in Richard J. Donovan Correctional Facility at Rock Mountain near San Diego, California.