Gordon was born in Brooklyn in New York City on December 2, 1922. Reared by his father in dire poverty, Gordon grew up during the Great Depression. He left school in the eighth grade, went to work in construction and demolition, and then joined the New Deal agency, the Civilian Conservation Corps, in which he participated in various public works projects. After the United States entered World War II in 1941, Gordon enlisted in the U.S. Army, in which he served for two years and received an undesirable discharge. Gordon was in southern California where he and a cohort attempted to rob a bar and its patrons with a pistol. He was shot in the stomach by one of the officers making the arrest. He was arrested for armed robbery and served five years in San Quentin Prison, where he furthered his education by reading nearly every book in the library.

Gordon started his career on the stage and worked with such luminaries as Edward G. Robinson and Tyrone Power. He was soon discovered by a Hollywood agent in a Los Angeles production of Darkness at Noon. Over the course of his career Gordon would appear in more than 170 film and television productions from the early 1950s to the mid-1990s.

In 1954 Gordon portrayed the outlaw Bill Doolin, a native Arkansan who founded the Wild Bunch gang and operated primarily in Kansas, on the syndicated television series Stories of the Century, starring and narrated by Jim Davis.

In 1955 he was cast on the ABC religion anthology series Crossroads in the role of Sergeant Leroy in "All My Love". In 1958 he appeared as Joe Brock in the episode "Desert Fury" of CBS's Tales of the Texas Rangers, a children's program. That same year Gordon was cast as Zip Wyatt in "Three Wanted Men" of Rex Allen's syndicated Western series Frontier Doctor. He also played a gunslinging professional killer in the pilot for the television version of Gunsmoke; but many changes were later instituted on the series, such as the marshal's office and Long Branch Saloon looking markedly different and the relationship between Matt Dillon and Kitty being subtly more formal as well, so the episode was buried deep in the season in the hope that viewers would not notice, which apparently worked.

Gordon portrayed sympathetic parts when called upon to do so, including his performances in the Western Black Patch (1957), a film that he wrote, and in Roger Corman's civil rights drama The Intruder (1962), opposite a young william Shatner.

Gordon appeared in multiple roles on Robert Stack's 1959 ABC crime drama The Untouchables. Gordon also guest-starred on the ABC/WB western series The Alaskans opposite Roger Moore. He was cast as Damian in the 1961 episode "Million Dollar Suit" of the ABC/WB crime drama The Roaring 20s. He also appeared on the NBC Western series Empire and Laredo.

Perhaps Gordon's single most memorable film scene occurred in McLintock! (1963), during which John Wayne knocks him down a long mudslide after uttering the famous line "Somebody oughta belt you but I won't! I won't! The hell I won't." Another notable role was in the 1966 western The Night of the Grizzly opposite Clint Walker, one of the very few actors who could match Gordon's intense screen presence regarding physical size and strength. Gordon played bounty hunter Cass Dowdy, who had a soft spot for his enemy's son but would, as one character said, "...hunt anything for a price, man or animal." Somehow, Gordon managed to make his character as sympathetic as he was frightening, and in his final scene he gives his life to save the boy.

In 1965, he was cast as the troublemaker Bender in the syndicated western series Death Valley Days in the episode "No Gun Behind His Badge", a dramatization of the Abilene, Kansas, marshal Thomas J. Smith, depicted in the segment by Ronald W. Reagan. The episode also starred Michael Witney as Wild Bill Hickok, who succeeded Smith as marshal.

Gordon was also a prolific Screenwriter and Novelist. Usually credited as "Leo V. Gordon", he wrote dozens of scripts for television series and movies, sometimes writing a good role for himself. His first successful film script, The Cry Baby Killer, featured a young and unknown Jack Nicholson. Among the more notable feature films he wrote was You Can't Win 'Em All (1970) starring Tony Curtis and Charles Bronson. He also wrote the screenplay and appeared in Tobruk (1967), which starred Rock Hudson and George Peppard and was directed by Arthur Hiller. As a television Screenwriter he wrote nearly 50 scripts apiece for Bonanza, Cheyenne and Maverick, in the episodes in which he was not a guest star. In the 1970s he appeared frequently as well on the popular police drama Adam-12, another show that he often scripted. Gordon once told an interviewer that because of his imposing size he never felt he was fully accepted as a screenwriter: "Writing is more rewarding than acting, but look at my face. Nobody believes I’m a Writer. I should be 5' 8", 142 pounds, wear patches on my elbows and horn-rimmed glasses and smoke a pipe. That's a Writer!" In addition to his script work for films and television Gordon wrote or co-wrote several novels, including the historical Western Powderkeg.

Gordon's final role was as Wyatt Earp in a 1994 episode of the television series The Young Indiana Jones Chronicles. He also appeared in the film Maverick that same year with Mel Gibson, Jodie Foster, and James Garner.

In contrast to his screen persona Gordon was a quiet, thoughtful and intelligent man who generally avoided the Hollywood spotlight. He was widely regarded by his fellow actors and his Directors as a well-prepared professional. In 1997 he received the Golden Boot Award for his many years of work in westerns. In accepting the award the actor simply flashed a smile for his fans and remarked, "Thank God for typecasting!"

After struggling with a brief illness, Gordon died of cardiac failure in his sleep, aged78, at his home in Los Angeles, California, on December 26, 2000. His ashes and those of his wife, who died in 2004, are interred together in a memorial display in a columbarium at the Hollywood Forever Cemetery in Los Angeles.