In 1993, he was nominated for the Ian Charleson Award and won the Manchester Evening News Award for his performance in The Comedy of Errors at the Royal Exchange Theatre, Manchester. In March 2011 he won a Laurence Olivier Award for Best Actor in a Supporting Role in a revival of the play After the Dance by Terence Rattigan.

Scarborough made his big-screen debut in 1994 in The Madness of King George. His other movie appearances have included Sweet Revenge, Gosford Park, Vera Drake, Notes on a Scandal, The History Boys, Elizabeth: The Golden Age, The King's Speech, and Les Misérables.

One of Scarborough's early television roles was J.G. Quiggin in the 1997 adaptation of A Dance to the Music of Time. He was also seen in the BBC series Let Them Eat Cake with Dawn French and Jennifer Saunders. He played "Bouffant", the gay couturier/servant/confidante to Saunder's character, the "Comtesse de Vache". His other appearances on television have included productions such as Cranford, the television film Into the Storm, Psychoville and as Charlie in the BBC comedy series Miranda. He has appeared in three episodes of Midsomer Murders, playing separate characters in series 5, series 10, and series 18. He is also the voice of Abney in the BBC Children's Television programme Abney & Teal. He voiced Benjamin Bunny in The World of Peter Rabbit and Friends.

In 2007, he played the character Harry Hepple in an episode of BBC New Tricks series 4.4 Nine Lives.

On stage, Scarborough has made frequent appearances at the Donmar, the Almeida, and in the West End and he has appeared in twenty productions for the Royal National Theatre. On 19 June 2011, he took part in The Barn Theatre presents... with Elliot Brown at the Barn Theatre, Welwyn Garden City Most recently he has appeared in Hedda Gabler at the Old Vic, again with frequent collaborator Sheridan Smith. In 2013, he played a role in Darkside, Tom Stoppard's radio drama based on Pink Floyd's album The Dark Side of the Moon.

In 2012 he appeared in the ITV series Mrs Biggs with fellow Gavin & Stacey star Sheridan Smith, playing the titular character's father Bernard (the real-life father of Charmian Brent, ex-wife of Ronnie Biggs). He played an alien known as Kahler-Jex in an episode of Doctor Who entitled "A Town Called Mercy", which aired on 15 September 2012.

In 2015, Scarborough appeared as Clive Trueman in the BBC TV series Father Brown episode 3.15 "The Owl of Minerva". Scarborough also appeared in the Starz sitcom Blunt Talk as Harry, valet to Walter Blunt, a role which he reprised in the second season a year later.

In 2016, Scarborough appeared as Colin in the Channel Four sitcom Crashing and as Tony Pitt in the ITV series Midsomer Murders episode 18.4 "A Dying Art"

In 2017, Scarborough will appear in the film adaptation of Ian McEwan's On Chesil Beach as Lionel Mayhew.