Lederman was born in New York City, New York, the son of Minna (née Rosenberg) and Morris Lederman, a laundryman. Lederman graduated from the James Monroe High School in the South Bronx. He received his bachelor's degree from the City College of New York in 1943, and received a Ph.D. from Columbia University in 1951. He then joined the Columbia faculty and eventually became Eugene Higgins Professor of Physics. In 1960, on leave from Columbia, he spent some time at CERN in Geneva as a Ford Foundation Fellow. He took an extended leave of absence from Columbia in 1979 to become Director of Fermilab. Resigning from Columbia (and retiring from Fermilab) in 1989 to teach briefly at the University of Chicago, he then moved to the physics department of the Illinois Institute of Technology, where he served as the Pritzker Professor of Science. In 1991, Lederman became President of the American Association for the Advancement of Science.

After three years in the U.S. Army during World War II, he took up physics at Columbia University, and received his Masters in 1948. Lederman began his Ph.D research working with Columbia's Nevis synchro-cyclotron, which was the most powerful particle accelerator in the world at that time. Dwight D. Eisenhower, then the President of Columbia University, and Future President of the United States, cut the ribbon dedicating the synchro-cyclotron in June 1950. These atom smashers were just coming of age at this time and created the new discipline of particle physics.

After receiving his Ph.D and then becoming a faculty member at Columbia University he was promoted to full professor in 1958.

Among his achievements are the discovery of the muon neutrino in 1962 and the bottom quark in 1977. These helped establish his reputation as among the top particle physicists.

In 1977, a group of physicists, the E288 experiment team, led by Leon Lederman announced that a particle with a mass of about 6.0 GeV was being produced by the Fermilab particle accelerator. The particle's initial name was the greek letter Upsilon (${\displaystyle \Upsilon \,}$). After taking further data, the group discovered that this particle did not actually exist, and the "discovery" was named "Oops-Leon" as a pun on the original name and Lederman's first name.

In 1988, Lederman received the Nobel Prize for Physics along with Melvin Schwartz and Jack Steinberger "for the neutrino beam method and the demonstration of the doublet structure of the leptons through the discovery of the muon neutrino". Lederman also received the National Medal of Science (1965), the Elliott Cresson Medal for Physics (1976), the Wolf Prize for Physics (1982) and the Enrico Fermi Award (1992).

A former President of the American Physical Society, Lederman also received the National Medal of Science, the Wolf Prize and the Ernest O. Lawrence Medal. Lederman served as President of the Board of Sponsors of The Bulletin of the Atomic Scientists. He also served on the board of trustees for Science Service, now known as Society for Science & the Public, from 1989 to 1992, and was a member of the JASON defense advisory group.

In 1995, he received the Chicago History Museum "Making History Award" for Distinction in Science Medicine and Technology.

He is known for his sense of humor in the physics community. On August 26, 2008 Lederman was video-recorded by a science focused organization called ScienCentral, on the street in a major U.S. city, answering questions from passersby. He answered questions such as "What is the strong force?" and "What happened before the Big Bang?".

In The God Particle he wrote, "The history of atomism is one of reductionism – the effort to reduce all the operations of nature to a small number of laws governing a small number of primordial objects." And this was the quest he undertook. This book shows that he pursued the quark, and hoped to find the Higgs boson. The top quark, which he and other physicists realized must exist according to the standard model, was, in fact, produced at Fermilab not long after this book was published.

He has three children with his first wife, Florence Gordon, and now lives with his second wife, Ellen (Carr), in Driggs, Idaho. In May 2015 his Nobel Prize gold medal was sold for US$765,000. According to his wife Ellen, they faced the uncertainty of medical bills related to his dementia diagnosis. She noted, "It's really hard. I wish it could be different. But he's happy. He likes where he lives with cats and dogs and horses. He doesn't have any problems with anxiety, and that makes me glad that he's so content."