Whitney was often a supporting character actor credited at least in the top ten actors appearing in several Hollywood classic feature films, such as Destination Tokyo (1943), Action in the North Atlantic (1943), Mr. Skeffington (1944), Murder, He Says (1945) (in which he played a dual role), The Big Heat (1953), In the Heat of the Night (1967), The Ballad of Cable Hogue (1970), and others before becoming well known for his work in television.

From the late 1950s, Whitney played character roles in many television series, including nine appearances on ABC's The Rifleman. One of his roles in The Rifleman was "Mail Order Groom" (1960), an episode in which he portrays John Jupiter, a man of great physical strength who must exert patience while he is harassed by two townsmen, played by John Anderson and Sandy Kenyon, who had quarreled with Whitney's intended spinster bride, Isabel Dent, played by Alice Backes (1923–2007).

In the 1958-1959 season, Whitney had a co-starring role as Buck Sinclair, a former sergeant of the Union Army, in all thirty-nine episodes of the ABC western series The Rough Riders. He also guest-starred on the ABC/Warner Brothers western series Colt .45. He performed the part of Ralph in the 1958 episode "Mantrap," with Don "Red" Barry cast as Percival. He played United States Secret Service agent Gunnerson in the episode "Savage Hills" on the ABC/WB series, Maverick. In 1960, in the episode "Surface of Truth" of another ABC/WB western series, Lawman, Whitney played Lucas Beyer, a crude white man who has lived for ten years with the Cheyenne Indians. Beyer accidentally kills his wife in a fit of rage and runs into Laramie, Wyoming, to escape the wrath of her pursuing brother. Marshal Dan Troop (John Russell) uses an old Indian "truth" method to try to determine what happened.

In 1960, in "The Longest Rope", the season premiere episode of the ABC/WB western series Cheyenne, Whitney was cast as the cruel, corrupt and entrenched Sheriff Hugo Parma of the community of High Point, where series protagonist Cheyenne Bodie (Clint Walker) had lived for a time in his early teens. In that episode's storyline, Bodie returns for a sentimental visit to High Point, only to find himself a successful but reluctant write-in candidate for sheriff against Parma. Donald May and Merry Anders also guest-starred in the episode as young adults from Bodie's past in High Point.

Whitney made three guest appearances on the CBS courtroom drama series Perry Mason: in 1961 he performed as the character Roger Gates in "The Case of the Pathetic Patient"; in 1962, as prison escapee Stefan "Big Steve" Jahnchek in "The Case of the Stand-in Sister"; and in 1965, as Captain Otto Varnum in "The Case of the Wrongful Writ."

Whitney also appeared on such series as The Public Defender, Gunsmoke, Adventures of Superman, City Detective, Fury, My Friend Flicka ("A Case of Honor"), The Californians, Sheriff of Cochise, Behind Closed Doors, Northwest Passage, Tate, Tombstone Territory (episode "Apache Vendetta"), Johnny Ringo, Riverboat, Bourbon Street Beat, The Alaskans, Overland Trail (as Governor Sutcliff in episode "First Stage to Denver"), The Rebel, The Islanders, Adventures in Paradise, Straightaway, Wagon Train, The Untouchables, Bonanza, The Monkees, Petticoat Junction (as Jasper Tweedy, in the 1969 episode: "Sorry Doctor, I Ain't Taking No Shots"), Green Acres, The Beverly Hillbillies, Rawhide (episode "Incident of the Music Maker"), and Hawaii Five-O. In addition, Whitney played a French partisan fighter named Massine in the 1963 episode "Thunder from the Hill" of ABC's military drama Combat!, as well as the character Caleb Calhoun in a 1964 episode of the Daniel Boone series. Then, in 1965, Whitney was cast as Judge Roy Bean in the episode "A Picture of a Lady" on the syndicated television series Death Valley Days, then hosted by Ronald W. Reagan. Francine York played Lily Langtry in that episode, and Paul Fix assumed the role of as the judge's friend, Doc Lathrop.

Peter Whitney's final role on television was that of a grave robber in Writer Rod Serling's series Night Gallery, in a 1972 episode segment titled "Deliveries in the Rear."