In 2003, Delaney toured the UK supporting Jerry Sadowitz, performing at venues including Manchester University and Shepherd's Bush Empire, London. His first Edinburgh Festival Fringe show, Purist, won generally very positive reviews, including four stars from comedy website Chortle, and The Independent newspaper. Delaney had two of his gags included in the top ten of the third annual Dave Award for the Funniest Joke of the Fringe, the only Comedian to do so. Following an increased profile due to appearances on Mock the Week and Dave's One Night Stand, Delaney toured Purist in 2013.

He began a relationship with fellow stand up Comedian Sarah Millican in 2006. The couple moved in together in 2013 and married in December 2013.

In July 2012 he appeared on Mock the Week. Some residents of Jersey were offended when he joked that people from Jersey were "trying to shake off their tax avoidance tag and get back to their traditional reputation as Nazi sympathisers." The BBC, however, reiterated that Mock the Week contains irreverent humour and that the comment was "obviously tongue-in-cheek".

Contributing to the topical podcast No Pressure to be Funny, in May 2013, he described himself as a "right-wing libertarian".