Elliott was born in Winchester, Massachusetts, the son of Gail Marguarite (née Brackett), a needleworker, and Fred Russell Elliott, who worked in insurance. Bob Elliot served In the U.S. Army in Northern Europe in WWII. On radio, he appeared in programs with his long-time partner Ray Goulding. These were in different series and time slots over decades, beginning in the late 1940s at Boston's WHDH radio when the two were first paired for Matinee with Bob and Ray. The team’s ersatz advertisements included exhortations on behalf of the Monongahela Metal Foundry (“Steel ingots cast with the housewife in mind”), Einbinder Flypaper (“The flypaper you’ve gradually grown to trust over the course of three generations”) and Height Watchers International.

Elliott married Jane Underwood in 1943. They divorced in 1953 having no children. Bob and Ray Writer Raymond Knight died in 1953. In 1954, Elliott married Knight's widow, Lee Elliott (née Peppers). They were married for 58 years until her death in 2012. They had two sons, Chris Elliott and Bob Elliott Jr., and one daughter, Amy Andersen. They adopted two children, Colony Elliott Santangelo and Shannon Elliott. They had 11 grandchildren and five great-grandchildren. In 1987, his oldest granddaughter, Abby Elliott, was born.

Elliott appeared on radio with Garrison Keillor in The American Radio Company of the Air. On television, Elliott and Goulding hosted Bob and Ray show from 1951 to 1953. He appeared on a number of other television programs, including Happy Days; Newhart; and Bob & Ray, Jane, Laraine & Gilda in 1979 (with Goulding, Jane Curtin, Laraine Newman and Gilda Radner); The David Steinberg Show; and Saturday Night Live. In 1982, Elliot was in Author! Author! as Patrick Dicker. He would star in made-for-TV-Movie's such as Between Time and Timbuktu and FDR: A One Man Show. Elliot made television commercials just as Goulding and Elliot had years earlier when they provided the voices for Bert and Harry Piel, the animated spokesmen for a New York brewing company.

In 1970, the duo debuted in The Two and Only on Broadway. Bob and Goulding worked together up until Goulding’s death in 1990. In 1990, Elliot portrayed a bank guard in Quick Change. In 1990, he portrayed "Fred Peterson" in the television series Get a Life, which starred Chris as his son. Four years later, the elder Elliott appeared in the Tim Burton production Cabin Boy, playing Chris' father again. In 2004, he appeared in a skit on the Air America radio program The O'Franken Factor.

Abby and her younger sister Bridget (nicknamed "Bridey") were raised in Wilton, Connecticut. In 1989, Elliott co-authored son Chris's mock autobiography, Daddy's Boy: A Son's Shocking Account of Life with a Famous Father.

Elliott died in Cundy's Harbor, Maine on February 2, 2016, from throat cancer at the age of 92.