James was born in Redlands, California, the son of Ida Mae (née Buckelew) and Jimmy James. He spent his early years in Beaumont, California, where his parents owned and operated a movie theater; James had said, "My story is like Cinema Paradiso. Every night in my life since I was 2 years old... I ran movies". After graduating from high school in 1962, James attended San Diego State University as a Theater Arts major. Migrating to New York, James immersed himself in the theatre scene, taking on bit roles here and there.

In 1975, James landed a small role in the made-for-television film, The Kansas City Massacre, playing John Dillinger gang member Homer Van Meter. Higher profile roles followed in 1976, with his casting in Nickelodeon and Harry and Walter Go to New York. James also appeared in the acclaimed television miniseries Roots and popular 1970s shows such as Gunsmoke, The Incredible Hulk, Mork and Mindy, Chico and the Man, and CHiPs.

James' career took a sudden upturn in the early 1980s with several sharply defined character roles in films such as Southern Comfort and 48 Hrs. (which were both directed by Walter Hill), but it was his performance as Leon Kowalski in the 1982 film Blade Runner that gave him his greatest, most lasting fame. Even though his memorable performance threatened to typecast the intense yet versatile actor as a movie villain for the remainder of the decade, James continued to pile up a prolific acting resume, playing significant roles in Enemy Mine, Flesh + Blood, A Breed Apart, Silverado, Armed and Dangerous, Red Heat, Steel Dawn, Red Scorpion, Tango & Cash and Showdown (portraying an obnoxious high-school vice-principal, Kowalski, whose name was probably an inside joke inspired by Blade Runner). James continued his strong work on the small screen as well, with guest spots in Benson, Quincy, M.E., The A-Team, Little House on the Prairie, The Dukes of Hazzard, Matlock, Miami Vice, Sledge Hammer!, and Dynasty. In the 1990s, he appeared in Highlander: The Series, and as Sheriff Bowman in the Millennium season 2 episode "Luminary". He lent his voice to the character of Parasite in Superman: The Animated Series. In 1982, he was in the made-for-TV-movie Hear No Evil as Billy Boy Burns.

His only starring role was in the low-budget 1989 supernatural horror film The Horror Show (aka House III), where he played serial killer "Meat Cleaver Max" Jenke. This was his all-time favorite role. The character was expected to spin off into a series, but there were no other movies with this character. James was relieved as he said he didn't want to make a career out of something so schlocky.

In 1994, he played a grouchy sponsor who became a victim of the gruesome goings-on during a 1939 radio show in the film Radioland Murders. Another of his most memorable roles came near the end of his career, as the amiable General Munro in The Fifth Element (1997). Two months before his death, James reprised his role as Parasite in Superman 64, a video game based on the critically acclaimed Superman: The Animated Series.

James died in 1999 from a heart attack in Malibu, California. He appeared in five feature films that were released posthumously. The last of these was Phoenix Point (2005).

The motion picture The King Is Alive (2000) was dedicated to him.