Born in Rocky Mount, North Carolina, Nelson appeared in several television series in the late 1940s, Captain Video and His Video Rangers and The Aldrich Family among them. He was cast in his first Broadway show, Seventeen, a musical adaptation of the Booth Tarkington novel that opened at the Broadhurst Theatre on June 21, 1951 and ran 182 performances.

In 1960, Nelson was cast in an off-Broadway show entitled The Fantasticks, which eventually became the world's longest-running musical with 17,162 performances. In 1962, he was hired to understudy Anthony Newley in Stop the World - I Want to Get Off when it transferred from the West End, eventually assuming the lead role when the star departed the show. From there, he went to another London import, Half a Sixpence, in 1965.

In 1968, Nelson accepted the lead in the controversial and groundbreaking off-Broadway production of The Boys in the Band, the first play to explore the milieu of gay life in New York City in a verbally frank manner. He and the rest of the cast went on to appear in the 1970 film version directed by william Friedkin.

Also in 1970 Nelson returned to Broadway in the lead role in Lovely Ladies, Kind Gentlemen, a musical adaptation of The Teahouse of the August Moon. It was a critical and commercial disaster, closing after only 19 performances. After settling in England in 1971, he played opposite Cleo Laine in "Showboat" and "Colette" on the West End stage, and appeared in "Annie" and "42nd Street." In 1974, he played a leading role in the highly successful revue Cole at London's Mermaid Theatre. In 1985 he played sinister nuclear Entrepreneur Jerry Grogan in Edge of Darkness, the seminal BBC thriller series written by Troy Kennedy Martin, and had roles in the Clive Barker horror films Hellraiser (1987) and Nightbreed (1990).

Nelson spent much of the later part of his career in small roles on television and in movies. Nelson died in 1993 of AIDS-related complications in London.