Anderson was born in New York City, the son of Swedish immigrants. He studied physics and engineering at Caltech (B.S., 1927; Ph.D., 1930). Under the supervision of Robert A. Millikan, he began investigations into cosmic rays during the course of which he encountered unexpected particle tracks in his (modern versions now commonly referred to as an Anderson) cloud chamber photographs that he correctly interpreted as having been created by a particle with the same mass as the electron, but with opposite electrical charge. This discovery, announced in 1932 and later confirmed by others, validated Paul Dirac's theoretical prediction of the existence of the positron. Anderson first detected the particles in cosmic rays. He then produced more conclusive proof by shooting gamma rays produced by the natural radioactive nuclide ThC'' (Tl) into other materials, resulting in the creation of positron-electron pairs. For this work, Anderson shared the 1936 Nobel Prize in Physics with Victor Hess.

Also in 1936, Anderson and his first graduate student, Seth Neddermeyer, discovered the muon (or 'mu-meson', as it was known for many years), a subatomic particle 207 times more massive than the electron, but with the same negative electric charge and spin 1/2 as the electron, again in cosmic rays. Anderson and Neddermeyer at first believed that they had seen the pion, a particle which Hideki Yukawa had postulated in his theory of the strong interaction. When it became clear that what Anderson had seen was not the pion, the Physicist I. I. Rabi, puzzled as to how the unexpected discovery could fit into any logical scheme of particle physics, quizzically asked "Who ordered that?" (sometimes the story goes that he was dining with colleagues at a Chinese restaurant at the time). The muon was the first of a long list of subatomic particles whose discovery initially baffled theoreticians who could not make the confusing "zoo" fit into some tidy conceptual scheme. Willis Lamb, in his 1955 Nobel Prize Lecture, joked that he had heard it said that "the finder of a new elementary particle used to be rewarded by a Nobel Prize, but such a discovery now ought to be punished by a 10,000 dollar fine."

Anderson spent all of his academic and research career at Caltech. During World War II, he conducted research in rocketry there. He was elected a Fellow of the American Academy of Arts and Sciences in 1950. He died on January 11, 1991, and his remains were interred in the Forest Lawn, Hollywood Hills Cemetery in Los Angeles, California. His wife Lorraine died in 1984.