Born on the 16 of March 1928 in Darmstadt, Germany, Böhm was the son of Austrian Conductor Karl Böhm and German-born Soprano Thea Linhard. He was an only child, and spent his youth in Darmstadt, Hamburg and Dresden. In Hamburg he attended elementary school and the Kepler-Gymnasium (a grammar school). Faked papers (claiming he had a lung disease) enabled him to emigrate to Switzerland in 1939, just around the beginning of World War II, where he attended the Lyceum Alpinum Zuoz, a boarding school. In 1946, he moved to Graz with his parents, where he graduated from high school the same year. He originally intended to become a Pianist but received poor feedback when he auditioned. His father urged him to study English and German language and literary studies, followed by studies of history of arts for one semester in Rome after which he quit and returned to Vienna to take acting lessons with Prof. Helmuth Krauss.

From 1948 to 1976 Böhm acted in about 45 films and also in theatre. With Romy Schneider, he starred in Sissi (1955), the first of a film trilogy, as Emperor Franz Joseph, with Schneider as his wife, Empress Elisabeth of Austria. The role for a time limited him to one specific genre as an actor, but Böhm's best known English language film was a dramatic change of image. In Peeping Tom (1960) he played the psychopath Mark Lewis. Director Michael Powell cast him in the role because he felt Böhm might understand the character's experience of having an overbearing father. The film's initial rejection hurt both the actor and Powell, for Powell professionally as well as emotionally, but it is now regarded as a classic. One unusual aspect of the casting is that Böhm displayed a significant German accent throughout the movie, though the character had been born and raised in England to, probably, an English father, as played for short bits by Powell without an accent.

Böhm's first wife was Elisabeth Zonewa. The marriage lasted from 1954 to 1957 and resulted in the birth of his daughter Sissy (born 1955). In her autobiography Sissy Böhm would later accuse her by-then-deceased parents of sexual child abuse.

Böhm was married from 1958 to 1962 to Gundula Blau, and next from 1963 to 1980 to Polish Actress Barbara Kwiatkowska-Lass. His fourth and last marriage was with Almaz Böhm, a native of Ethiopia in 1991. They had two children, Nicolas (born 1990) and Aida (born 1993). Böhm had five more children from previous marriages, among them the Actress Katharina Böhm (born 1964). In February 2013 it was reported that he was suffering from Alzheimer's disease, he lived in Grödig near Salzburg until his death in May 2014.

Briefly, in the early 1960s, Böhm worked in the American film and television industry. He played Jakob Grimm in the MGM-Cinerama spectacular The Wonderful World of the Brothers Grimm and Ludwig van Beethoven in the Walt Disney film The Magnificent Rebel. The latter film was made especially for Walt Disney's Wonderful World of Color television anthology series, but it was released theatrically in Europe. He appeared in a villainous role as the Nazi-sympathizing son of Paul Lukas in the MGM film Four Horsemen of the Apocalypse (all 1962), a remake of the 1921 silent Rudolph Valentino film.

During 1974 and 1975, Böhm appeared prominently in four consecutive films from prolific New German Cinema Director Rainer Werner Fassbinder: Martha, Effi Briest, Faustrecht der Freiheit (Fox and His Friends), and Mutter Küsters' Fahrt zum Himmel (Mother Küsters' Trip to Heaven).

Böhm's voice acting work included narrating his father's 1975 recording of Peter and the Wolf by Sergei Prokofiev and in 2009 providing the German voice for Charles Muntz, villain in Pixar's tenth animated feature Up.

On the ZDF show Wetten, dass..? in 1981 Böhm raised DM 1.2 million for people in Africa. He bet that "not every third viewer would donate one Mark, one Swiss franc or seven Austrian schillings for needy people in the Sahel zone". In November 1981, Böhm founded Menschen für Menschen (Humans for Humans) and involved in charitable work in Ethiopia. He largely retired from acting in the 1980s for his project. Until today, Menschen für Menschen built over 400 schools, 2000 fountains and over 5 million people benefit from their work.

Böhm received honorary Ethiopian citizenship in 2003. In 2007 he was awarded the Balzan Prize for Humanity, Peace and Brotherhood among Peoples. In 2011 Karlheinz Böhm and his wife Almaz were awarded the Essl Social Prize for the project Menschen für Menschen.