On leaving school, he trained as a quantity surveyor and later volunteered for the Royal Air Force in 1941 but was not accepted because of a perforated eardrum. He then enlisted with the Black Watch and he served for five years during the Second World War, which included three years spent in India.

After being demobbed, Mackay began training as an actor at RADA. His first work was with the Citizens' Theatre, Glasgow, where he performed in nine seasons between 1949 and 1958. He also worked at the Royal Lyceum Theatre, Edinburgh before gaining notice at the Arts Theatre Club, London, where in 1960, he played the part of Oscar in The Naked Island, a play about POWs in Singapore.

In 1962, he appeared at the same theatre, in Russian Playwright Maxim Gorki's classic The Lower Depths for the Royal Shakespeare Company. He then acted with the Old Vic company and the National Theatre, performing in such productions as Peer Gynt and The Alchemist. Other roles for the RSC included Mr Squeers in Nicholas Nickleby and the drunken gaoler in Die Fledermaus.

Before coming to prominence in Porridge, he played DI Inman in Special Branch (1969–71). He also appeared in Z-Cars. He appeared as an RAF Psychiatrist in an episode of Some Mothers Do 'Ave 'Em and as a Doctor in Doctor at Large in 1971.

He was cast as misguided scientist Doctor John Quinn in the 1970 Doctor Who story Doctor Who and the Silurians and was later seriously considered by Producer Barry Letts to play the Fourth Doctor when Jon Pertwee announced he was leaving the role in 1973.

He played a regular officer running a training course in the Dad's Army episode "We Know Our Onions" (1973), and a Doctor in "The Miser's Hoard" (1977).

He often stayed true to his Scottish roots, acting in productions such as Three Tales of Orkney and The Master of Ballantrae, and as former Prime Minister Bonar Law in the 1981 TV series The Life and Times of David Lloyd George. He played the Captain in the British version of the children's series, Fraggle Rock (1983–1984). In one of his last performances, Mackay portrayed an art forger in the Lovejoy episode "Death and Venice".

He was married to Irish Actress Sheila Manahan. He did much work for the Glasgow children's charity Child and Family Trust. He was appointed an OBE in 1984 and greatly enjoyed oil painting.

Mackay died on 6 June 1987, aged 64 from stomach cancer. His body was buried at East Sheen Cemetery, Greater London, England. After his wife Sheila died in 1988, her body was interred in the same grave.