Simon was born in 1908 in Mansfield in Richland County in north central Ohio, where he was an all-state high school basketball champion in the 1920s. He played basketball in one-on-one games well into his forties. Before he entered acting, he was a traveling salesman. He initially thought that acting would help him to overcome his natural shyness, but he enjoyed appearing on stage and later film so much that he decided to make a career as an actor. His first professional job was in Marc Blitzstein's No For an Answer in which he displayed his song-and-dance abilities. He was also involved with The Cleveland Play House and then with The Actors Studio, through which he became an understudy to Lee J. Cobb for the lead in Arthur Miller's Death of a Salesman.

Simon had two daughters, Barbara Ann Simon Callet (born 1941) and Susan Simon Thompson, and two sons, Robert Louis Simon (born 1950) and James A. Simon (born 1951). Simon's webpage does not include the name of his spouse(s).

Simon appeared on Broadway in Clifford Odets's play, Clash by Night. In 1949, he succeeded Lee J. Cobb as Willy Loman in Death of a Salesman.

His first film appearances were as Inspector Foley in Where the Sidewalk Ends (1950) and as a Psychiatrist in Bright Victory (1951). His first television role was at the age of 43 in 1952 as Captain Scott in the episode "Woman with a Sword" on Hallmark Hall of Fame anthology series. Simon appeared as Ackerman in the 1954 film Rogue Cop. Between 1951 and 1954, he appeared on The Philco Television Playhouse and Justice.

In 1955, he appeared on television in episodes of Medic and Alfred Hitchcock Presents as well as such feature films as Chief Crazy Horse, Seven Angry Men, and The Court-Martial of Billy Mitchell. Actress Elizabeth Montgomery, who would later play Simon's daughter-in-law, Samantha, on Bewitched, made her film debut in The Court-Martial of Billy Mitchell.

There were few television westerns in which Simon did not guest star. From 1956 to 1970, he appeared in Broken Arrow, Disneyland, Dick Powell's Zane Grey Theater, Laramie, Black Saddle, Law of the Plainsman, Johnny Ringo, Cheyenne, and The Dakotas, Wichita Town, The Man From Blackhawk, The Texan, Tombstone Territory, Tate, and Shotgun Slade, Stagecoach West, Bat Masterson, Lawman, Klondike, and Frontier Circus, Have Gun - Will Travel, Wagon Train, The Legend of Jesse James, The Road West, Gunsmoke, Laredo, The Virginian, Bonanza, and The Guns of Will Sonnett.

Simon guest-starred three times on Perry Mason., including the role of murderer Edward Bannister in the 1958 episode, "The Case of the Desperate Daughter." Simon appeared as Harvey, friend of the main character Paul Driscoll in the 1963 The Twilight Zone episode "No Time Like the Past". In 1965, he appeared in episodes of Slattery's People, Voyage to the Bottom of the Sea, and Dr. Kildare.

During the 1960s, Simon performed as well in dramatic roles in several films, such as The Spiral Road (1962), Captain Newman, M.D. (1963), and Fate Is the Hunter (1964). In 1966, he starred too as Mr. Rellik in the Highway Safety Films' production The Third Killer. His role was that of a "Death" salesman charged with three accounts, including traffic fatalities.

Simon appeared in such programs as Crusader, Route 66, Dante, The DuPont Show with June Allyson, Johnny Midnight, Straightaway, The Roaring 20s, Sea Hunt and State Trooper. In 1961 and 1962, he guest starred on episodes of Ripcord, The Dick Powell Show, The Lloyd Bridges Show, Cain's Hundred, The Defenders and Sam Benedict.

In 1962, Simon played the part of Mackie in the episode "House of the Hunter" on CBS's Rawhide. The same year he also portrayed Handy Strong in the feature film The Man Who Shot Liberty Valance.

He appeared in A New Kind of Love (1963) starring Paul Newman and Joanne Woodward, as "Cervantes" in The Reluctant Astronaut (1967), and as a Doctor in Private Duty Nurses (1971). He appeared in a 1970 episode of Love, American Style, in a 1971 episode of Nichols, starring James Garner, and a 1973 episode of The Partridge Family. In 1973, he made three guest appearances as General Maynard M. Mitchell on M*A*S*H.

From 1969 to 1985, Simon appeared in Marcus Welby, M.D., The Mod Squad, The Interns, Barnaby Jones, Hawaii Five-O, Cannon, Ellery Queen, Columbo, McCloud, Quincy M.E., Eight Is Enough, and The Feather and Father Gang. His last television appearance was in a 1985 episode of Airwolf.

Simon died of a heart attack in Tarzana, California on November 29, 1992, three days before what would have been his 84th birthday. He is interred at Oakwood Memorial Park Cemetery in Chatsworth, California.