Anderson was born in Indianapolis, Indiana and grew up in Urbana, Illinois. He graduated from University Laboratory High School in Urbana in 1940. Afterwards, he went to Harvard University for undergraduate and graduate work, with a wartime stint at the U.S. Naval Research Laboratory in-between. In graduate school he studied under John Hasbrouck van Vleck.

From 1949 to 1984 he was employed by Bell Laboratories in New Jersey, where he worked on a wide variety of problems in condensed matter physics. During this period he developed what is now called Anderson localization (the idea that extended states can be localized by the presence of disorder in a system); invented the Anderson Hamiltonian, which describes the site-wise interaction of electrons in a transition metal; proposed symmetry breaking within particle physics (this played a role in the development of the Standard Model and the development of the theory behind the Higgs mechanism, which in turn generates mass in some elementary particles); created the pseudospin approach to the BCS theory of superconductivity; made seminal studies of non-s-wave pairing (both symmetry-breaking and microscopic mechanism) in the superfluidity of He3; and helped found the area of spin-glasses. He was elected a Fellow of the American Academy of Arts and Sciences in 1963.

He was awarded the Oliver E. Buckley Condensed Matter Prize in 1964, the Nobel Prize in Physics in 1977 and was elected a Foreign Member of the Royal Society (ForMemRS) in 1980. He was awarded the National Medal of Science in 1982.

From 1967 to 1975, Anderson was a professor of theoretical physics at Cambridge University. In 1977 Anderson was awarded the Nobel Prize in Physics for his investigations into the electronic structure of magnetic and disordered systems, which allowed for the development of electronic switching and memory devices in computers. Co-researchers Sir Nevill Francis Mott and John van Vleck shared the award with him. In 1982, he was awarded the National Medal of Science. He retired from Bell Labs in 1984 and is currently Joseph Henry Professor of Physics, Emeritus at Princeton University.

Anderson has also made conceptual contributions to the philosophy of science through his explication of emergent phenomena. In 1972 he wrote an article called "More is Different" in which he emphasized the limitations of reductionism and the existence of hierarchical levels of science, each of which requires its own fundamental principles for advancement.

A 2006 statistical analysis of scientific research papers by José Soler, comparing number of references in a paper to the number of citations, declared Anderson to be the "most creative" amongst ten most cited physicists in the world.