Mustin married Frances Robina Woods in 1915. They remained married until Woods' death in 1969. The couple had no children.

Mustin was one of the 110 original founders of the Pittsburgh chapter of the Lions Club which was established in 1921. He served as one of the Presidents and remained active in the club for the remainder of his life.

The next year, Mustin co-starred in the television film version of Miracle on 34th Street, starring Sebastian Cabot, and had an uncredited role in the Disney television film Now You See Him, Now You Don't. Mustin's last continuing role was on the television series Phyllis, in which he played the suitor, and later husband, of Sally "Mother" Dexter, a role he played until shortly before his death.

In addition to guest-starring roles, Mustin also had recurring roles on several television shows during the 1950s and 1960s. In 1955, he played the role of "Foley" in The Great Gildersleeve. From 1957 to 1958, he appeared as Mr. Finley on Date with the Angels. In 1957, he made his first appearance as "Gus the Fireman" on Leave It to Beaver. Mustin would continue in the role until 1962, making a total of 15 appearances on the show. In 1960, he made his first guest appearance on The Andy Griffith Show as Jud Fletcher. He appeared in the role until 1966. He also portrayed "Old Uncle Joe" on two episodes of The Lucy Show in 1967. The following year, Mustin guest starred as "Grandpa Jenson" in three episodes of Petticoat Junction.

In addition to his extensive work in television, Mustin also appeared in numerous films. He made his film debut at the age of 67 in Detective Story, in 1951. He followed this with roles in Talk About a Stranger (1952), The Silver Whip (1953), Half a Hero (1953), She Couldn't Say No (1954), The Desperate Hours (1955), Man with the Gun (1955), Storm Center (1956), and The Sheepman (1958).

In the 1960s and 1970s, Mustin appeared in The Adventures of Huckleberry Finn (1960), Hemingway's Adventures of a Young Man (1962), Twilight of Honor (1963), What a Way to Go! (1964), The Misadventures of Merlin Jones (1964), Sex and the Single Girl (1964), The Cincinnati Kid (1965), Cat Ballou (1965) (uncredited as a former gunfighter "Old ... Old ... ?" ), The Ghost and Mr. Chicken (1965), The Adventures of Bullwhip Griffin (1967), Speedway (uncredited) (1968), The Shakiest Gun in the West (1968), The Great Bank Robbery (1969), Hail, Hero! (1969), and Skin Game (1971). In 1974, Mustin portrayed "Uncle Jeff" in the musical film Mame, starring Lucille Ball and Bea Arthur. He also had a small role in Herbie Rides Again, also released in 1974. The next year, he appeared as "Regent Appleby" in The Strongest Man in the World. His final film role came in 1976 in the Western film Baker's Hawk, starring Clint Walker and Burl Ives.

Known for his quick wit and song-and-dance abilities, Mustin was a frequent guest on The Tonight Show Starring Johnny Carson during the 1970s. From 1971 to 1976, he appeared in five episodes of All in the Family (his first appearance as a night watchman, and an additional four appearances in a recurring role as "Justin Quigley").

In 1971, Mustin co-starred in the Sketch comedy show The Funny Side. Hosted by Gene Kelly, the series featured an ensemble cast of five married couples that dealt with various issues through comedy sketches and song-and-dance routines. Mustin was cast opposite Queenie Smith as "the elderly couple". The series debuted on NBC in September 1971 and was canceled in January 1972.

Mustin guest starred as Jethroe Collins, a relative of a Jesse James victim in the "Bobby's Hero" episode of the Brady Bunch during the 1972–73 season.

On January 28, 1977, Mustin died at Glendale Memorial Hospital in Glendale, California, less than two weeks before his 93rd birthday. He was buried in Forest Lawn - Hollywood Hills Cemetery.

In 2000 TVLand created a series of commercials celebrating the often seen but little known by name character actors who regularly appeared in their shows, with Mustin being featured in one.