Sylvester was born in Oakland, California, the son of Italian immigrant Joseph Silvester (1899-1941) and his American wife, Gertrude Bailey Silvester (1898-1957). He served in the U.S. Navy during the Second World War. He settled in Britain after the war to pursue his interest in professional acting. He became a staple of British B films at a time when American and Canadian actors were much in demand in order to give indigenous films some appeal in the United States. While in the UK, he married British Actress Veronica Hurst.

As a result, he gained top billing in one of his very first films, House of Blackmail (1953), directed by the veteran filmmaker Maurice Elvey, for whom he also made What Every Woman Wants the following year. He also starred in such minor films as A Stranger Came Home (1954, for Hammer), Dublin Nightmare (1958), Offbeat (1960), Information Received (1961), Incident at Midnight, Ring of Spies and Blind Corner (all 1963). There were also lead roles in four British horror films — Gorgo (1960), Devil Doll (1963), Devils of Darkness (1964) and The Hand of Night (1966). Among his many television credits were a 1959 BBC version of Shakespeare's Julius Caesar (playing Mark Antony), The Saint, The Baron, The High Chaparral, Harry O, Danger Man, The Six Million Dollar Man and Quincy, M.E.

His other film work included an uncredited role in You Only Live Twice (1967). In spite of his prominent role for Kubrick, Sylvester's film career never prospered and he appeared mainly on television and in small roles in movies such as Busting (1974), The Hindenburg (1975) and Heaven Can Wait (1978). He retired from acting in the early 1980s and died in Sacramento, California in on January 25, 1995, six days before his 73rd birthday.