The actor became a star of the Hungarian stage and screen in the 1910s and 1920s. At the beginning of the 1920s, he moved to Vienna, where he appeared in Hermann Leopoldi's Kabarett Leopoldi-Wiesenthal. In the 1930s, he was, next to Hans Moser, the most significant representative of the Wiener Film, the Viennese light romantic comedy genre. He also appeared in Berlin.

He appeared in Familientag im Hause Prellstein (1927), Ihre Majestät die Liebe (1929, which was remade in Hollywood as Her Majesty Love, with W.C. Fields in Sakall's role) and Two Hearts in Waltz Time (1930). For a brief period during this time, he ran his own production company.

When the Nazis came to power in Germany in 1933, Sakall was forced to return to Hungary. He was involved in over 40 movies in his native land. When Hungary joined the Axis in 1940, he left for Hollywood with his wife. Many of Sakall's close relatives later died in Nazi concentration camps, including all three of his sisters and his niece, as well as his wife's brother and sister.

Sakall began a Hollywood career that included "an endless succession of excitable theatrical impresarios, lovable European uncles and befuddled shopkeepers". His first American film role was in the comedy It's a Date (1940) with Deanna Durbin. The first big hit of his American career was Ball of Fire (1941) with Gary Cooper and Barbara Stanwyck. Later, he signed a contract with Warner Bros., where he had a number of other small roles, including one in Yankee Doodle Dandy (1942) with James Cagney.

Later the same year, at the age of 59, he portrayed his best remembered character, Carl the head waiter in Casablanca (1942). Producer Hal B. Wallis signed Sakall for the role three weeks after filming had begun. When he was first offered the part, Sakall hated it and turned it down. Sakall finally agreed to take the role provided they gave him four weeks of work. The two sides eventually agreed on three weeks. He received $1,750 per week for a total of $5,250. He actually had more screen time than either Peter Lorre or Sydney Greenstreet.

Sakall appeared in 30 more movies after this, including Christmas in Connecticut (1945), reuniting with Barbara Stanwyck. Sakall appeared in four films released in 1948: the drama Embraceable You, followed by April Showers, Michael Curtiz's Romance on the High Seas (Doris Day's film debut), and Whiplash.

He was in four top movies in 1949. First Sakall played Felix Hofer in Doris Day's second film, My Dream Is Yours. Later that year, he supported June Haver and Ray Bolger in Look for the Silver Li Ning. Next, he played Otto Oberkugen in In the Good Old Summertime, with Judy Garland and Van Johnson. This was a remake of Ernst Lubitsch's The Shop Around the Corner (1940). Finally, Sakall was given the principal role of Songwriter Fred Fisher in Oh, You Beautiful Doll, though top billing went to June Haver.

Sakall appeared in nine more movies during the 1950s, two of them musicals with Doris Day, playing J. Maxwell Bloomhaus in Tea for Two (1950) and Adolph Hubbell in Lullaby of Broadway (1951). His other roles included: Poppa Schultz in the Errol Flynn western Montana (1950); Miklos Teretzky in the June Haver musical The Daughter of Rosie O'Grady (also 1950); Don Miguel in the Randolph Scott western Sugarfoot; Uncle Felix in the musical Painting the Clouds with Sunshine (1951) with Virginia Mayo, and one of the episodes in the movie It's a Big Country (also 1951) featuring Gene Kelly, Van Johnson, Gary Cooper, Janet Leigh, Fredric March and Ethel Barrymore. His last movie was The Student Prince (1954) in which he played Joseph Ruder.

Sakall died of a heart attack in Hollywood on 12 February 1955, shortly after filming The Student Prince, ten days after his 72nd birthday. He is buried in the Garden of Memory in Forest Lawn Memorial Park in Glendale, California.