Herman was born into a Jewish family in New York City and raised in Washington Heights, Manhattan. He graduated from the Fiorello H. LaGuardia High School of Music & Art and Performing Arts in 1987, then attended the acting program at SUNY Purchase. Herman appeared in the films Born on the Fourth of July, Lost Angels, and Let It Be Me; but he is probably best known for his role as Michael Bolton in Mike Judge's Office Space. Earlier, Herman had joined the cast of John Leguizamo's House of Buggin' in 1994; however, that show was short-lived. When Fox decided to mount a Sketch comedy show to replace House of Buggin', Herman was kept on as an original MADtv cast member.

Herman was one of nine original cast members of MADtv when the series debuted in 1995. Unlike the other original repertory and featured players, he was one of the few cast members who came to the show with an established acting career. Herman's MADtv characters included Mike Lawson (Incredible Findings), Generation X anchorman Marsh (X News), and concerned father Joel Linder. Former castmate Artie Lange referred to Herman as "a brilliant, talented actor" and called him "one of my best friends in show Business."

Herman was a repertory performer on the show for two and a half seasons. He left MADtv in the middle of the show's third season (1997–98) to pursue a movie career and other projects.

Although he is primarily a television actor, Herman has appeared in several films including Dude, Where's My Car? and Fun with Dick and Jane. Herman had a prominent role in the 1999 comedy Office Space, where he played a gangsta rap-loving nerd unfortunately named Michael Bolton who must endure questions about his relationship to the famous singer. In 2006, he appeared in another Mike Judge film, Idiocracy.

Herman is also known for his work as a voice actor in cartoons and video games, notably in Futurama, where he provides the voices of many recurring characters (including Scruffy the Janitor, Roberto the psycho robot, Mayor Poopenmeyer, and Professor Farnsworth's rival Ogden Wernstrom), and King of the Hill, for which he voiced Anthony "Twig Boy" Page (the social worker from the pilot episode who accused Hank of beating his son and from season two's "Junkie Business", where he forces Strickland Propane to hire a drug addict who has his addiction listed as a disability), Hank's childhood friend, Eustace, and other incidental characters. His other voice work includes Family Guy, American Dad!, Father of the Pride (in which he voices Roy Horn), and the Jak and Daxter video game series. He starred as Ubuntu Goode (and other supporting voices) in The Goode Family. He also provides the voice for Mr. Frond (and other supporting voices) in Fox's Bob's Burgers and from 2012 to 2015 worked on the Comedy Central series Brickleberry as self-absorbed park ranger Steve Williams.