He studied for ten years at the Moscow Conservatory, preparing for a pianist's career. In 1960, however, he met Andrei Tarkovsky and co-scripted his movie Andrei Rublev (1966).

His first full-length feature, The First Teacher (1964), was favourably received in the Soviet Union and screened by numerous film festivals abroad. His second film, Asya Klyachina's Story (1967), was suppressed by Soviet authorities. When issued twenty years later, it was acclaimed as his masterpiece. Thereupon, Konchalovsky filmed adaptations of Ivan Turgenev's A Nest of Gentle Folk (1969) and Chekhov's Uncle Vanya (1970), with Innokenty Smoktunovsky in the title role.

Konchalovsky has been married five times. His first wife was Irina Kandat. His second wife was Kazakh Actress Natalia Arinbasarova, with whom he has one son: Yegor, born January 15, 1966. His third wife was Viviane Godet, with whom he has a daughter, Alexandra Mikhalkova, born October 6, 1971. His fourth wife is Irina Ivanova, with whom he has two daughters: Nathalia and Elena. His fifth wife is Russian Actress Julia Vysotskaya; they have been married since 1998 and have two children: Masha (1999) and Piotr (2003).

His film The Postman's White Nights won the Silver Lion at the 71st Venice International Film Festival.

In 1979 he was a member of the jury at the 11th Moscow International Film Festival. His epic Siberiade upon its 1979 release was favourably received at Cannes and made possible his move to the United States in 1980.

His most popular Hollywood releases are Maria's Lovers (1984), Runaway Train (1985), based on a script by Japanese Director Akira Kurosawa, and Tango & Cash (1989), starring Sylvester Stallone and Kurt Russell. In the 1990s, Konchalovsky returned to Russia, although he occasionally produced historical films for U.S. television, such as his adaption of The Odyssey (1997) and the award-winning remake, The Lion in Winter (2003).

Konchalovsky's full-length feature, House of Fools (2003), with a cameo role by Bryan Adams as himself, set in a Chechen psychiatric asylum during the war, won him a Silver Lion at the Venice Film Festival.

His film, The Nutcracker in 3D had its American release on November 24, 2010 and premiered in Estonia and Russia on January 1, 2011. The film had a reported $90,000,000 budget and brought in a total of $65,944 in its U.S opening weekend. The film was also critically derided, with Roger Ebert stating in his review of the film, "From what dark night of the soul emerged the wretched idea for 'The Nutcracker in 3D?' Who considered it even remotely a plausible idea for a movie?"

In 2016 Paradise directed by him won the Silver Lion at the 73rd Venice International Film Festival. It was selected as the Russian entry for the Best Foreign Language Film at the 89th Academy Awards.