After several early television roles, including The Henderson Kids alongside Kylie Minogue, he attracted notice in his breakout film, The Year My Voice Broke (1987), winning him the Australian Film Institute Award for Best Supporting Actor. His next major role was in The Big Steal (1990), and Spotswood (1992) co-starring with Anthony Hopkins; this was followed in 1996 by Cosi and Idiot Box. In 2000, he was in two contrasting films, the Australian Mullet and the Hollywood Vertical Limit.

Whilst filming Adoration, Mendelsohn took time free to pose for Artist Sally West, whom he had been neighbours with in Bondi in the 1990s. The resulting portrait has been nominated for several awards.

In 2005, he was preparing to play Mark Antony in the Sydney Theatre Company-produced Julius Caesar, and he was in the Terrence Malick-directed film The New World.

In 2007, Mendelsohn starred in the third season of the TV series Love My Way and in 2008, he appeared in Baz Luhrmann's Australia and filmed the 10-part Melbourne series Tangle, which premiered on Showcase in 2009. In 2009, he appeared in the American science fiction film Knowing directed by Alex Proyas. The same year, Mendelsohn starred as Ned in Beautiful Kate, directed by Rachel Ward, opposite Bryan Brown and Rachel Griffiths.

In October 2009, he featured in the Australian series Who Do You Think You Are?, tracing the ancestry of his paternal grandfather, who was from a Jewish family, as well as convicts on his mother's side. Searching for a connection to Composer Felix Mendelssohn, which was eventually dismissed, he discovered links to 19th century Prussia. His ancestors were among the first Prussian Jews to be naturalised in Schneidemühl in the province of Posen, now Piła in modern Poland. He also has Greek, German, and British Isles ancestry.

An impression of Mendelsohn is featured repeatedly on the podcast by Australian Sketch comedy group, Aunty Donna. Mendelsohn is played by Broden Kelly and is often an imitation of his character from the 2010 film, Animal Kingdom

Mendelsohn married British author Emma Forrest in June 2012. He has two daughters, one with Forrest, born in 2014, and one from a previous relationship. He divorced from Forrest in 2016.

In 2014, Mendelsohn joined the cast of Bloodline, a Netflix original from the creators of Damages. The first season premiered on the site on 20 March 2015 and was well received. Mendelsohn's performance on the series was lauded by critics, resulting in a Primetime Emmy Award win as well as a Golden Globe Award nomination. In September 2016, Netflix announced that the show had been cancelled, and that it would end after its third season in 2017. Season 3 of Bloodline received negative reviews and Mendelsohn appeared in two episodes of it.

In December 2016, he had a supporting role in the Star Wars franchise's Rogue One, portraying villain Director Krennic. He called the opportunity to act in a Star Wars film "a childhood dream come true". Mendelsohn also narrated the virtual band Gorillaz' fourth album Humanz.