Erwin was born in Honey Grove, Texas to Lee Eugene and Ida Mae (née Lindsey) Erwin. He had a sister named Mary Gene Cosper. He attended San Angelo College before earning his bachelor's degree in journalism at the University of Texas, graduating in 1935. He completed a masters of theater arts at California's Pasadena Playhouse in 1941. After serving as a Captain in the United States Army Air Corps in World War II, Erwin returned to Hollywood to resume his acting career. His first film role was in the 1942 film You're in the Army Now.

Erwin began his theatrical career as Ventriloquist Edgar Bergen's stage manager for Bergen's 1941 tour of the country. Erwin dryly recalled, "I was in charge of the dummies."

Erwin was married to Actress and Journalist Fran MacLachlan Erwin from 1948 to her death in 1995. They lived in the Hollywood Hills and had two sons, Michael and Timothy, and two daughters, Lindsey and Kelly.

His television credits were far more numerous in the 1950s, having appeared in such television series as I Love Lucy, Crusader, Trackdown, Colgate Theatre, Perry Mason and The Rifleman. In the 1960s, Erwin appeared in television series such as: The Andy Griffith Show, Mister Ed, Maverick, The Twilight Zone, 87th Precinct, My Three Sons, The Fugitive, Leave it To Beaver and Mannix.

In the 1970s, 1980s and 1990s, he appeared in Barnaby Jones, Cannon, Gunsmoke, Married With Children, E/R, Highway to Heaven, Who's The Boss?, Growing Pains, Full House, The Golden Girls, Moonlighting, and Star Trek: The Next Generation. In Star Trek: The Next Generation, Erwin played Dr. Dalen Quaice, a friend and mentor of Dr. Beverly Crusher. He was the first character to disappear in the episode "Remember Me".

Erwin also appeared in numerous films by John Hughes, with cameos in Planes, Trains and Automobiles (1987), She's Having a Baby (1988), Home Alone (1990), and Dennis the Menace (1993). Hughes often paired him with Billie Bird playing his wife. His later film career included roles in Invitation to Hell (1984), Naked Gun 33⅓: The Final Insult (1994), Things to Do in Denver When You're Dead (1995), Menno's Mind (1997), Chairman of the Board (1998), Forces of Nature (1999), Inferno (1999) and A Crack in the Floor (2001).

In the Seinfeld episode ("The Old Man"), for which Erwin received an Emmy nomination for outstanding guest actor, he played Sid Fields, who participates in the Foster-A-Grandpa Program, which pairs him with Jerry Seinfeld. Erwin's crochety, aggressive, foul-mouthed character ensures that the relationship is doomed from the beginning. Erwin later reunited with Michael Richards when he guest-starred on the short-lived The Michael Richards Show. In the 2000s, Erwin appeared on Monk, The West Wing, The King of Queens, Everwood and My Name Is Earl.

In the late 1950s, Erwin was in such films as Man from Del Rio (1956) and The Night Runner (1957), before playing Jack Nicholson's father in The Cry Baby Killer, Nicholson's first starring role in 1958. The long out-of-print film was released on DVD on November 22, 2006. He had credited small roles in films such as The Christine Jorgensen Story (1970), How Awful About Allan (1970), Candy Stripe Nurses (1974) and Tarantulas: The Deadly Cargo (1977), before he co-starred alongside Christopher Reeve and Jane Seymour in the 1980 romantic fantasy Somewhere in Time as Arthur Biehl, the Grand Hotel's venerable bellman, and attended annual reunions of cast, crew, and fans of the film at the Grand Hotel on Mackinac Island, Michigan.

Erwin died from natural causes at his home in Studio City, California on December 29, 2010. He lived near the production lot where Seinfeld was filmed.