After a series of short term "fill-in" jobs that included catering, farming and journalism, he secured a position at the Chichester Theatre where he worked as an assistant stage manager and later as a stand-in Producer. He auditioned in 1968 for a production of Alan Bennett's new play, Forty Years On, which featured John Gielgud as the headmaster of a British public school during the First World War period. Andrews was cast as Skinner, one of twenty schoolboys. In 1974 he had a role in the production of Upstairs Downstairs as Lord Robert, Marquis of Stockbridge. In 1975 he had a leading role in the Spanish film Las adolescentes (The Adolescents), opposite Koo Stark.

In June 1977, he was cast in the role of Bodie in the ITV series The Professionals. However, after three days of filming, the creator and Producer Brian Clemens felt that the chemistry between Andrews and Martin Shaw (Doyle) did not work and that "the pair did not have the required undercurrent of menace to carry off the concept". Lewis Collins replaced Andrews in the part. Following that, in 1979, Andrews was the main star of the ITV television series Danger UXB, in which he played a British bomb disposal officer in the London Blitz. The series first aired in the United States in 1979 on Masterpiece Theatre.

His subsequent work includes the leading role of Lord Sebastian Flyte in Brideshead Revisited. In 1982, he won a Golden Globe and BAFTA TV Award for his performance and was nominated for an Emmy Award. In the United States, Andrews is best known for his portrayal of the titular character in Ivanhoe as well as that of Sir Percy Blakeney in the 1982 film The Scarlet Pimpernel.

He played Professor Higgins in a stage version of My Fair Lady (2001), and Count Fosco in Andrew Lloyd Webber's The Woman in White.

Andrews survived a case of water intoxication in 2003. The condition, known as hyponatraemia, is caused by the dilution of sodium in the body. It has similar symptoms to dehydration, such as headaches, nausea and cramps. Whilst performing as Henry Higgins in My Fair Lady he would consume up to eight litres of water a day. He lost consciousness and spent three days in intensive care.

He was the narrator for a 21st Anniversary BBC Radio 2 special broadcast of Cameron Mackintosh's musical Les Misérables, sung by the then West End cast at the Mermaid Theatre in London on Sunday 8 October 2006. Andrews appeared as Prime Minister Stanley Baldwin in the 2010 film The King's Speech, for which he won a SAG Award along with Helena Bonham Carter, Jennifer Ehle, Colin Firth, Michael Gambon, Derek Jacobi, Guy Pearce, Geoffrey Rush and Timothy Spall.