Kelly was born in Manchester, Lancashire on 19 December 1943. He attended the Liverpool Collegiate School and was a chorister at Liverpool Cathedral, where he showed early acting talent by reciting monologues. He worked for three years in the Civil Service in Liverpool before training as an actor at the London Academy of Music and Dramatic Art. After graduating in 1967, he appeared in repertory theatres around the UK.

His early roles included playing a film Director in Tiffany Jones (1973) and appearances in two of the later Carry On films, Carry On Dick and Carry On Behind. He then had a significant role in the British sitcom Porridge as the affable and high spirited Slade Prison inmate Bunny Warren who relies on fellow prisoners to read his letters from home and blames his incarceration on his inability to read. In the sitcom 'Allo 'Allo! he played German officer Captain Hans Geering, leaving after the third season (something he later said that he regretted doing). He played the law-stationer Mr. Snagsby in the 1985 BBC adaptation of Bleak House. In On the Up he played Dennis Waterman's chauffeur and he appeared in We'll Think of Something as Les Brooks. In the BBC 1994 TV serial 'Martin Chuzzlewit' Sam Kelly was Mr. Mould, the undertaker. From 1995 to 2003, he played Barbara's husband Ted in the British sitcom Barbara.

He worked with Director Mike Leigh on several occasions, including Grown-Ups (1980), Topsy-Turvy (1999), All or Nothing (2002) and on stage at the National Theatre in Grief (September 2011).

In the 1990s, he co-starred in the comedy television series Haggard and in the miniseries Martin Chuzzlewit. He also performed in H.M.S. Pinafore with the new D'Oyly Carte Opera Company in 2002. In 1998, Kelly appeared in an episode of the first series of Cold Feet, playing Algernon Gifford.

In 1996, Kelly appeared at the National Theatre in Helen Edmundson's adaptation of Leo Tolstoy's War and Peace. He played Bernard in Holding On (1997) and Carl Langbehn in the five-part television drama Christabel (1988). He appeared in Midsomer Murders "Down Among the Dead Men" as Jack Fothergill and as the eccentric impoverished ghostwriter, Majors, in the Inspector Morse episode "Second Time Around".

In 2004, he appeared in EastEnders playing Stan Porter, and he also appeared in the comedy series Black Books as the father of Manny. In 2006 he appeared as the villain Guy Carse in New Tricks.

Kelly starred in Jean-Paul Sartre's play Kean alongside Antony Sher at the Theatre Royal, Bath, and in the West End in May 2007. In December 2007, a car struck and injured Kelly in the West End. He was to have performed in the Doctor Who episode ("Midnight") filmed in that month, but the role was taken by David Troughton instead. Kelly though did act in the Doctor Who audio dramas "The Holy Terror" and "Return to the Web Planet" by Big Finish Productions.

In 2008, he guest starred in the Sapphire and Steel audio drama Remember Me. In November 2008, he starred in the title role of Christopher Reason's radio dramatisation of Jaroslav Hašek's The Good Soldier Švejk, broadcast on BBC Radio 4.

Kelly was in a long-term relationship with Journalist and psychotherapist Grace Pieniazek until her death in 2009.

On 23 July 2010, Kelly starred as Martin in the sitcom My Family and in the Heartbeat episode "Deadlier Than the Male". He also starred in When We Are Married at London's Garrick Theatre and in Mike Leigh's movie for the 2012 Olympics, A Running Jump. He opened at the National Theatre on 21 September 2011, in Mike Leigh's play, Grief, starring with Lesley Manville. His final film role was a cameo as a theatre actor in Mike Leigh's 2014 film Mr. Turner.

Kelly returned to Wicked as the Wizard on 18 November 2013.

It was reported in January 2014, that Kelly had temporarily departed the production due to ill health, and that his replacement from 17 February 2014, would be Martyn Ellis. He was admitted to a hospice on 13 June 2014 and died early the next morning aged 70 after a long battle with cancer.