Elam was born in Miami in Gila County in south central Arizona, to Millard Elam and Alice Amelia Kirby. His mother died in 1922 when Jack was two years old. By 1930, he was living with his father, older sister Mildred, and their stepmother, Flossie Varney Elam.

He grew up picking cotton and lost the sight in his left eye during a boyhood accident when he was stabbed with a pencil at a Boy Scout meeting. He was a student at both Miami High School in Gila County and Phoenix Union High School in Maricopa County, graduating from there in the late 1930s.

Jack Elam was married twice, first to Jean Hodgert from 1937 to her death in 1961, and then to Margaret Jennison from 1961 until his own death. Elam died of congestive heart failure in Ashland, Oregon in 2003, just a month before his 83rd birthday. He was survived by his wife Margaret; their daughter, Jacqueline; and his daughter and son from his previous marriage, Jeri and Scott.

In 1949, Elam made his debut in She Shoulda Said No!, an exploitation film in which a chorus girl's marijuana smoking ruins her career and drives her brother to suicide. He appeared mostly in westerns and gangster films playing villains.

Elam made multiple guest-star appearances in many popular Western television series in the 1950s and 1960s, including Gunsmoke, The Rifleman, Lawman, Bonanza, Cheyenne, Have Gun – Will Travel, Zorro, The Lone Ranger, The Rebel, F Troop, “Tales of Wells Fargo” and Rawhide. In 1961, he played a slightly crazed bus Passenger on The Twilight Zone episode "Will the Real Martian Please Stand Up?"

In 1963, Elam got a rare chance to play the good guy, Deputy U.S. Marshal and reformed gunfighter J. D. Smith, in the ABC/Warner Brothers series, The Dakotas, a western that was telecast for only nineteen episodes. He played George Taggart, a gunslinger-turned-marshal in the NBC/WB western series, Temple Houston, with Jeffrey Hunter in the title role. Elam got this part after James Coburn declined the role. Unfortunately for him, that series ran for only twenty-six weeks.

In 1966, Jack Elam co-starred with Clint Walker in the western The Night of the Grizzly. In 1968, Elam had a cameo in Sergio Leone's celebrated spaghetti western Once Upon a Time in the West. In that film he played one of a trio of gunslingers who were sent to kill Charles Bronson's character. Elam spent a good part of the scene trying to trap an annoying fly in his gun barrel. In 1967 Elam appeared in The Way West with Robert Mitchum, Richard Widmark and Kirk Douglas as the light hearted Preacher Weatherby taking part in a wagon train on the Oregon Trail. In 1969, he was given his first comedic role in Support Your Local Sheriff!, which was followed two years later by Support Your Local Gunfighter, both opposite James Garner. After his performances in those two films, Elam found his villainous parts dwindling and his comic roles increasing. (Both films were also directed by Burt Kennedy, who had seen Elam's potential as a Comedian and would direct him a total of 15 times in features and television.) In between those two films, he also played a comically cranky old coot opposite John Wayne in Howard Hawks's Rio Lobo (1970). In 1979 he was cast as the Frankenstein Monster in the CBS sitcom Struck by Lightning, but the show was cancelled after only three episodes. He then appeared in the role of "Hick Peterson" in a first-season episode of Home Improvement alongside Ernest Borgnine (Season 1, episode 20 "Birds Of A Feather Flock To Tim").

Elam played "Doctor Nikolas Van Helsing," an eccentric Doctor in the 1981 movie The Cannonball Run. Three years later, he returned in the same role in the film's sequel The Cannonball Run II.

In 1985, Elam played Charlie in The Aurora Encounter. During production, Elam developed what would become a lifelong relationship with an 11-year-old boy named Mickey Hays, who suffered from progeria. As shown in the documentary I Am Not a Freak viewers see how close Elam and Hays really were. Elam said, "You know I've met a lot of people, but I've never met anybody that got next to me like Mickey."

In 1986, Elam also co-starred on the short-lived comedy series Easy Street as Alvin "Bully" Stevenson, the down-on-his-luck uncle of Loni Anderson's character, L. K. McGuire. In 1988, Elam co-starred with Willie Nelson in the movie Where The Hell's That Gold?

In 1994, Elam was inducted into the Hall of Great Western Performers of the National Cowboy and Western Heritage Museum.