Born Károly Vidor to a Jewish family in Budapest, Hungary, he served in the Austro-Hungarian Army during World War I. He first came to prominence during the final years of the silent film era, working with Alex Korda among others. (He is not related to fellow Director King Vidor [1894–1982].)

Vidor went to Hollywood where he worked as Korda's assistant. He attracted acclaim for a low budget short he made in his spare time with his own money, The Bridge (1929). This led to a contract at Universal Pictures to work in the editorial department.

He did some uncredited directing on MGM's The Mask of Fu Manchu (1932). His first credited feature as Director was Sensation Hunters (1933) for Monogram Pictures. Vidor followed it with Double Door (1934) at Paramount.

Vidor accepted a contract to work at RKO Pictures. While there he directed Strangers All (1935), His Family Tree (1935), The Arizonian (1935), and Muss 'Em Up (1936).

Vidor went back to Paramount where he directed A Doctor's Diary (1937), The Great Gambini (1937), and She's No Lady (1937).

Frank Capra got Vidor in to do some second unit work on Mr Smith Goes to Washington (1939), which was running behind schedule. Vidor directed the scenes in Smith's home town. "I had more money to spend than I was ever given to make a feature picture," he later said.

Vidor was loaned out to Edward Small to direct My Son, My Son! (1940). Back at Columbia, he began to be assigned to more prestigious films like The Lady in Question (1940). He took over from James Whale on They Dare Not Love (1941) and did Ladies in Retirement (1941).

Vidor was loaned to Paramount to direct New York Town (1941) and RKO for The Tuttles of Tahiti (1942).

Vidor did Together Again (1944) then made a biopic of Chopin, A Song to Remember (1945), which was another big hit, and made a star of Cornel Wilde.

In 1946 Vidor sued Columbia, seeking to be released from his contract and $78,000 in damages. The case went to trial, where Vidor argued that he had been treated badly by Cohn, who swore at him. (Cohn admitted the swearing but said this was his way of expressing himself.) He also said Cohn would not loan him $25,000 to buy a new house and that Cohn made him cry twice by yelling at him. The judge ruled against Vidor, ordering him back to work.

He started directing The Guilt of Janet Ames (1947), but fought with Cohn and was replaced during filming by Henry Levin.

In 1948 Vidor announced he had purchased rights to Sirocco, a French Foreign Legion tale based on the novel Coup de Grace he wanted to make with Rita Hayworth and Humphrey Bogart. He was also preparing to do the film version of Born Yesterday and did a few days uncredited work on Under Cover Man.

In December 1949 Vidor signed a contract with MGM to direct The Running of the Tide which was never made.

Vidor went to Paramount with a project he had developed himself, Thunder in the East (filmed 1951, released 1952), which starred Alan Ladd.

Sam Goldwyn used him on Hans Christian Andersen (1952) which was a huge success. After filming Vidor optioned the novel SPQR and announced he would film it independently but it was not made.

At MGM Vidor did a musical with Elizabeth Taylor, Rhapsody (1954) and a biopic of Ruth Etting, Love Me or Leave Me (1955), a big success. Vidor directed Grace Kelly's last movie, The Swan (1956).

In 1956 he announced he had formed his own company, Aurora, to make The Life of Nijinsky.

Vidor went into partnership with Frank Sinatra and Joe E. Lewis to make a biopic of the latter, The Joker Is Wild (1957). Sinatra and Vidor were going to reunite on Kings Go Forth but then David O. Selznick hired Vidor to make the troubled A Farewell to Arms (1957), replacing John Huston.

Vidor's last film was an attempt to repeat the success of A Song to Remember, another biopic of a Composer, in this case Liszt: Song Without End (1960) (originally titled A Magic Flame). He died of a heart attack three weeks in to filming.