Hovis was born in Wapato, Washington, and moved to Houston, Texas, as a small child. As a youth, he was a singer, appearing on Arthur Godfrey's Talent Scouts. Hovis attended the University of Houston. During the mid-1950s, Hovis sang in nightclubs with groups including the Mascots, and the Bill Gannon Trio. He wrote songs and signed with Capitol Records, which released one album. His biggest song was We Could Have Lots of Fun.

Hovis began appearing in local theater productions. After some success, he moved to New York City in 1959 and appeared in Broadway revues such as From A to Z which showcased his singing and comedy talents.

Hovis moved to California in 1963 where he performed stand-up comedy and tried to break into television. In 1964, he was discovered by Andy Griffith's manager and was hired to appear on the TV series Gomer Pyle, U.S.M.C., where he played "Pvt. Larry Gotschalk". He also appeared on The Andy Griffith Show'As Gilly Walker..'.

In 1965, when another actor backed out of the television show Hogan's Heroes, Hovis was cast as "Sgt. Andrew Carter", a POW in a German prison camp who was an expert on explosives. In the pilot episode, Carter was a lieutenant and was only going to appear in that one episode. For the series he retained the character of Sgt. Carter, replacing a character played by Leonid Kinskey in the pilot. (Kinskey decided after the pilot that he didn't want to stay with a show that had actors pretending to be Nazis). In the series, Carter was of Sioux ancestry; Hovis himself was partly of Yakama Indian ancestry. Later, in an episode of the comedy Alice, Hovis played an American Indian police detective who arrests a fake American Indian conman.

While Hovis was a regular on Hogan's Heroes, he also did other work in the entertainment industry, including writing the screenplay for the 1966 spy-spoof Out of Sight. He also co-wrote Mitzi Gaynor's 1968 and 1969 television specials, and appeared in and wrote comedy bits for Rowan & Martin's Laugh-In.

In the mid-1970s, Hovis made a few appearances on the game show Match Game alongside his Hogan's Heroes castmate Richard Dawson. Later in the decade he produced and was a regular panelist on the game show Liar's Club.

Even before Hogan's Heroes was canceled in 1971, Hovis had already made appearances on other TV shows.

In the early 1980s, Hovis toured in the musical The Best Little Whorehouse in Texas as Melvin P. Thorpe. In 1982, Hovis was a writer/producer on the show So You Think You Got Troubles, which was hosted by actor/ventriloquist Jay Johnson. Later in the decade, Hovis teamed up with Gary Bernstein to form Bernstein-Hovis Productions, which produced the game shows Anything For Money, the original version of Lingo and the short-lived Yahtzee, a TV version of the classic dice game, for which Hovis also announced and served as a regular panelist.

Hovis was hired as a co-producer for the hidden-camera television show Totally Hidden Video, but was fired by Fox executives after Candid Camera creator Allen Funt filed a lawsuit alleging that Hovis had staged segments of the show's 1989 debut episode using paid actors.

Beginning in the 1990s, Hovis taught drama at Southwest Texas State University – now called Texas State University-San Marcos – in San Marcos, Texas.

Hovis died of esophageal cancer in Austin, Texas, on September 9, 2003. He was 67 years old.