Mueller-Stahl was born in Tilsit, East Prussia (now Sovetsk, Kaliningrad Oblast, Russia). His mother, Editta, was from an upper-class family and became a university professor in Leipzig. His father, Alfred Müller, was a bank Teller who changed the family's surname to "Mueller-Stahl". The rest of the family moved to Berlin while his father fought on the Eastern Front in World War II. Mueller-Stahl was a concert Violinist while he was a teenager and enrolled at an East Berlin acting school in 1952.

Mueller-Stahl was a film and stage actor in East Germany, performing such films as Her Third and Jacob the Liar. For that country's television, he played the main character of the popular series Das unsichtbare Visier from 1973–1979, a spy thriller program designed, in co-operation with the Stasi, as a counterpart to the James Bond films. After protesting against Wolf Biermann's denaturalisation in 1976 he was blacklisted by the government. Emigrating in 1980 to West Germany, he found regular work in films. These included Rainer Werner Fassbinder's Lola (1981) and Veronika Voss (1982), Andrzej Wajda's A Love in Germany (1984), Angry Harvest and Colonel Redl (both 1985), the latter about Alfred Redl.

Mueller-Stahl made his American film debut as Jessica Lange's character's father in Music Box (1989). He subsequently took character roles in Kafka by Steven Soderbergh and Night on Earth by Jim Jarmusch (both 1991). He is also remembered for his role as the Soviet general in charge of the occupied United States in the ABC television miniseries Amerika (1987). Mueller-Stahl's leading role in Avalon (1990) is also memorable.

Mueller-Stahl won the Silver Bear for Best Actor at the 42nd Berlin International Film Festival for his performance in Utz (1992). He was nominated for the Academy Award for Best Supporting Actor for his performance in Shine (1996). Mueller-Stahl was also in A Pyromaniac's Love Story (1995) and the 1997 remake of the movie 12 Angry Men. Conversation with the Beast (1996) was his first film as Director. In 1998, he played the German scientist and syndicate member, Conrad Strughold, in the feature film The X-Files. In 1999 he played the mastermind of a Criminal gang opposite Ray Liotta and Gloria Reuben in Pilgrim, also distributed under the title Inferno.

In the early 2000s, Mueller-Stahl received a positive response for his portrayal of Thomas Mann in a German film about the Mann family called Die Manns - Ein Jahrhundertroman. In 2004, Mueller-Stahl made a foray into American television, guest-starring in four episodes on the television drama series The West Wing as the Prime Minister of Israel. In 2006, he played the role of reclusive Russian Artist Nikolai Seroff in Local Color. He had a role in David Cronenberg's crime drama Eastern Promises (2007) and the thriller The International (2009), both of which co-starred British-Australian Actress Naomi Watts. In 2008, he won the Genie Award for Best Performance by an Actor in a Supporting Role for Eastern Promises, and Mueller-Stahl played the role of Cardinal Strauss, Dean of the College of Cardinals and the Papal conclave, in Angels & Demons (2009),

In 2011, he was awarded the Honorary Golden Bear at the 61st Berlin International Film Festival.