In 1902, when Herzfeld was 10 years old, he was enrolled in the private Gymnasium Schottengymnasium, which was run by the Benedictine Order of the Roman Catholic Church and had its name derived from the fact that the founders came from Scotland. He attended this school until 1910, when he began attending the University of Vienna to study physics and chemistry. In 1912, he took courses at the University of Zurich and the Eidgenössische Technische Hochschule Zürich (ETH). It was in Zurich he met Otto Stern, who was at the ETH; Herzfeld later credited conversations with Stern for his deeper understanding of thermodynamics. In 1913, he went to study at the University of Göttingen, after which Herzfeld returned to Vienna, and was granted his doctorate in 1914, under Friedrich Hasenöhrl, who had become Director of the Institute for Theoretical Physics, upon the suicide of Ludwig Boltzmann in 1906.

Upon receipt of his doctorate, Herzfeld volunteered for Service in the Austro-Hungarian Army. World War I broke out shortly thereafter and he served until 1918, rising to the rank of first lieutenant. Herzfeld’s thesis advisor Hasenöhrl was called to serve during World War I and was killed at the front. During his tenure in the military, Herzfeld published six papers on statistical mechanics applied to problems in physics and chemistry, especially to the structure of matter – gases, liquids, and solids.

After the War, Herzfeld returned to the University of Vienna, however, the University was in such dire financial straits that he moved to Munich in 1919, with the intent of studying analytical chemistry and getting a job in the German chemical industry, which had a highly respected reputation. First, he was an assistant at the physico-chemical laboratory of Kasimir Fajans at Ludwig Maximilians University of Munich (LMU). However, once there, he found the challenge of theoretical physics more to his liking. He became Privatdozent for theoretical physics and physical chemistry at LMU, and therefore was much more associated with Arnold Sommerfeld, who was ordinarius professor for theoretical physics and Director of the Institute for Theoretical Physics – a prominent organization for the study of atomic and molecular structure. From 1925, until he left LMU in 1926, he was extraordinarius professor of theoretical physics. During this time, Linus Pauling did postdoctoral studies with him, and he was the thesis advisor for Walter Heitler, who got his doctorate in 1926. In 1925, Herzfeld published his book on kinetic theory and statistical mechanics, which became a graduate-level textbook in German-speaking universities.

It was in 1926 that Herzfeld took a visiting professorship at the Johns Hopkins University in Baltimore, Maryland, which developed into a regular faculty position. During 1930 and 1932, he was a lecturer at Cooper Union and Fordham University in New York City.

While at Johns Hopkins, Herzfeld did considerable research with the Chemist Francis O. Rice, who joined the University as an associate professor the same year Herzfeld arrived. Their 1928 paper considered the role of molecular vibrations in the transfer of Energy between ultrasonic waves and gas molecules. At Johns Hopkins, Herzfeld worked with other European colleagues on the University’s physics faculty, namely James Franck and Maria Goeppert-Mayer, who were awarded Nobel Prizes in Physics in 1925 and 1963, respectively. Franck came to Johns Hopkins after he left Germany in 1933, where he had been ordinarius professor of experimental physics and Director of the Second Institute for Experimental Physics at the University of Göttingen and a close colleague of Max Born, who was Director of the Institute of Theoretical Physics at Göttingen. Goeppert-Mayer was a student of Born, and she joined the Johns Hopkins faculty in 1931. Goeppert-Mayer and Herzfeld published articles on states of aggregation and nuclear fusion reactions. Herzfeld coauthored articles with Franck on photosynthesis, one being after they had both left Johns Hopkins. John Archibald Wheeler, who became a prominent Physicist, took his PhD under Herzfeld in 1933.

In 1936, Herzfeld moved to The Catholic University of America in Washington, DC, where he remained until his death in 1978. He received emeritus status in 1969 and stayed active for the rest of his life.

In 1938, Herzfeld married Regina Flannery, who was an instructor of anthropology at Catholic University; by the time she retired in 1970, she had risen to professor and the first woman to head that department.

In the late 1940s, Herzfeld increased the attention at Catholic University to quantum-mechanical calculations on the electronic structure of polyatomic molecules, thus establishing a respected position for the University in this field.

In 1959, Herzfeld and Theodore A. Litovitz collaborated on a book, in part, summarizing Herzfeld’s thinking on ultrasonics over the 30 plus years since his article with F. O. Rice. In 1966, Herzfeld published a review article summarizing 50 years of developments in physical ultrasonics.

Reasons for Herzfeld leaving Johns Hopkins were described in a letter to Arnold Sommerfeld. A main reason was the dire financial situation at Johns Hopkins. However, there were other reasons as well. One being his relationship with R. W. Wood, a professor of experimental physics and chairman of the physics department, had deteriorated. Also, J. A. Bearden, another experimentalist, thought there was too much emphasis on theoretical physics and the number of German physicists in the small department was out of balance. Bearden also suspected that Herzfeld had brought Franck to Johns Hopkins to further Herzfeld’s ambitions to be department chairman. Finally too, Bearden thought Herzfeld had caused dissension in the department over his strong support to promote Göppert-Mayer from research associate in physics to a regular faculty appointment. While Herzfeld did receive offers from both Fordham University and Catholic University, neither was appealing as they did not have strong research departments. While talking the situation over with Isaiah Bowman, President of Johns Hopkins, it became clear that the financial difficulties at Johns Hopkins might require downsizing the physics faculty. With this in mind, Herzfeld accepted the offer from Catholic University. Herzfeld’s teaching responsibilities and salary at Catholic University were about the same as that at Johns Hopkins, but there were additional administrative duties, as he was also chairman of the physics department.