Mössbauer was born in Munich, where he also studied physics at the Technical University of Munich. He prepared his Diplom thesis in the Laboratory of Applied Physics of Heinz Maier-Leibnitz and graduated in 1955. He then went to the Max Planck Institute for Medical Research in Heidelberg. Since this institute, not being part of a university, had no right to award a doctorate, Mössbauer remained under the auspices of Maier-Leibnitz, who was his official thesis advisor when he passed his PhD exam in Munich in 1958.

On the suggestion of Richard Feynman, Mössbauer was invited in 1960 to Caltech, where he advanced rapidly from Research Fellow to Senior Research Fellow; he was appointed a full professor of physics in early 1962. In 1964, his alma mater, the Technical University of Munich (TUM), convinced him to go back as a full professor. He retained this position until he became professor emeritus in 1997. As a condition for his return, the faculty of physics introduced a "department" system. This system, strongly influenced by Mössbauer's American experience, was in radical contrast to the traditional, hierarchical "faculty" system of German universities, and it gave the TUM an eminent position in German physics.

In 1972, Rudolf Mössbauer went to Grenoble to succeed Heinz Maier-Leibnitz as the Director of the Institut Laue-Langevin just when its newly built high-flux research reactor went into operation. After serving a 5-year term, Mössbauer returned to Munich, where he found his institutional reforms reversed by overarching legislation. Until the end of his career, he often expressed bitterness over this "destruction of the department." Meanwhile, his research interests shifted to neutrino physics.

Rudolf Mössbauer was an excellent Teacher. He gave highly specialized lectures on numerous courses, including Neutrino Physics, Neutrino Oscillations, The Unification of the Electromagnetic and Weak Interactions and The Interaction of Photons and Neutrons With Matter. In 1984, he gave undergraduate lectures to 350 people taking the physics course. He told his students: “Explain it! The most important thing is, that you are able to explain it! You will have exams, there you have to explain it. Eventually, you pass them, you get your diploma and you think, that's it! – No, the whole life is an exam, you'll have to write applications, you'll have to discuss with peers... So learn to explain it! You can train this by explaining to another student, a colleague. If they are not available, explain it to your mother – or to your cat!”