McGraw married Freda Choy Kitt in 1938. They had a daughter.

He developed into a leading man, especially in the film noir genre during the late 1940s and early 1950s. His gravelly voice and rugged looks enhanced his appeal in that very stylistic genre.

McGraw made his first film in 1942 with a small, uncredited role in The Undying Monsterat Fox. He was in Tonight We Raid Calais (1942) and They Came to Blow Up America (1943) at the same studio, and also Two Tickets to London (1943), Destroyer (1943), Corvette K-225 (1943), The Mad Ghoul (1943), The Impostor (1944), and The Seventh Cross (1944).

His first notable role was in The Killers (1946), which opens with McGraw and fellow heavy william Conrad as the two hitmen who terrorize a small-town diner in their search for Burt Lancaster.

McGraw's parts remained small in T-Men (1947) for Anthony Mann, The Hunted (1948), Berlin Express (1948), Hazard (1948), and Blood on the Moon (1948). He had a bigger role in Once More, My Darling (1949), then went back to small parts in Reign of Terror (1949) and Border Incident (1949) for Mann, and The Story of Molly X (1949).

McGraw moved up to third billing in a noir, The Threat (1949). He played a cop in Side Street (1950) for Mann and a gangster in Ma and Pa Kettle Go to Town (1951).

McGraw was finally given a leading role in RKO's Armored Car Robbery (1950) directed by Richard Fleischer. He played a gangster in His Kind of Woman (1951) then had the lead in Roadblock (1951), as "Honest Joe", the insurance investigator turned thief by love.

The son of Francis Butters and Beatrice Crisp Butters, McGraw was born in Des Moines, Iowa. (A newspaper article published in 1951 says of McGraw, "He was born in New York City, but his parents moved to Akron, Ohio, when he was five years old.") In January 1932, he graduated from high school, later attending college for one semester.

Fleischer used McGraw in the lead of The Narrow Margin (1952) which has become a cult classic. He was a sergeant in One Minute to Zero (1952) and War Paint (1953) and was a villain in Thunder Over the Plains (1954).

McGraw starred as Mike Waring, the title character, in the 39-episode 1954–55 syndicated television series Adventures of the Falcon. The series updated the original Falcon premise to have Michael Waring as a secret agent in the Cold War. He also starred in the first television version of Casablanca (1955), taking Humphrey Bogart's role as Rick Blaine. Additionally, he had the role of Captain Hughes in The Smith Family. In 1963, McGraw played Dr. Simon Oliver on the pilot of Diagnosis: Danger, a medical drama.

McGraw is recognized with a star in the Television section of the Hollywood Walk of Fame, located at 6927 Hollywood Blvd., Hollywood, Los Angeles, California. It was dedicated February 8, 1960.

Charles McGraw died after slipping and falling through a glass shower door in his Studio City, California home on July 30, 1980, severing an artery in his arm. (A newspaper article published in 1981 gave August 2 as the date of McGraw's death.) His ashes were scattered in the Pacific Ocean.