Kaye was born Bernard Solomon (or Sholom) Kotzin on the last day of the First World War, at West 114th Street in the Morningside Heights section of Manhattan. His parents were first generation Jewish-Americans originally from Russia and Austria (Hungary). His father, David Kotzin, was a dress salesman, and the former Harriet "Hattie" Freundlish, was his mother. He was raised in the Far Rockaway section of Queens and later in The Bronx, where he acted in student productions at DeWitt Clinton High School and where he graduated in 1936.

In 1939 he won the Major Bowes Amateur Hour contest on radio where the prize included touring in vaudeville, where he was sometimes billed as an "Extra Padded Attraction". During the Second World War he joined the USO where he toured battle fronts and made his London debut performing with Bob Hope. After the war he continued to work in vaudeville and as Master of Ceremonies for the swing orchestras of Freddy Martin and Charlie Barnet.

His first wife was Jeanne Watson from Chicago, who was a clerical worker at the movie studios in the late 1950s. They were married in 1960 as the series Love and Marriage ended, but the couple divorced because of personal differences within a year of their marriage.

In the 1960-1961 season, Kaye appeared as Marty, the agent of aspiring Actress Eileen Sherwood, in the CBS sitcom My Sister Eileen, starring Shirley Bonne, Elaine Stritch, Jack Weston, Raymond Bailey, and Rose Marie.

In the 1960s, Kaye became well known as host of a weekly children's talent show, Stubby's Silver Star Show. During the 1962–1963 season, he was a regular on Stump the Stars. On April 14, 1963, he guest starred as "Tubby Mason" in NBC's Ensign O'Toole comedy series, starring Dean Jones.

From 1964 to 1965, he hosted the Saturday morning children's game show Shenanigans on ABC.

In 1965, Kaye appeared alongside Nat King Cole as a travelling musician in the western/comedy Cat Ballou, starring Jane Fonda and Lee Marvin. In 1969, Kaye played the role of Herman in the Universal musical film Sweet Charity directed by Bob Fosse which starred Shirley MacLaine in the title role. In that movie, he sang the song "I Love to Cry at Weddings".

Kaye's later stage productions included the 1974 Broadway revival of Good News, Man of Magic in London (with Stuart Damon as Harry Houdini), and his final Broadway show Grind co-starring Ben Vereen in 1985. He made a guest appearance in "Delta And The Bannermen", a story in the British science fiction series, Doctor Who in 1987. His last featured film role was as Marvin Acme in Robert Zemeckis's 1988 film Who Framed Roger Rabbit.

Kaye died on December 14, 1997, of lung cancer at the age of 79.

Directors viewed Kaye as a master of the Broadway idiom during the last phase of the musical comedy era. This was evidenced by his introduction of three show-stopping numbers of the era: “Fugue for Tinhorns” and “Sit Down, You’re Rockin’ the Boat” from Guys and Dolls (1950) and “Jubilation T. Cornpone” from Li'l Abner (1956). In 1953 he played in You Can't Run Away From It, a remake of It Happened One Night. Kaye is best known for defining the role of Nicely-Nicely Johnson in Guys and Dolls, first on Broadway and then in the film version. He also played Marryin' Sam in Li'l Abner, again on both stage and screen. In 1962, he played the title character in Michael Winner's The Cool Mikado.