Dolby was born in Portland, Oregon, the son of Esther Eufemia (née Strand) and Earl Milton Dolby, an Inventor. He attended Sequoia High School (class of 1951) in Redwood City, California. As a teenager in the decade following World War II, he held part-time and summer jobs at Ampex in Redwood City, working with their first audio tape recorder in 1949. While at San Jose State College and later at Stanford University (interrupted by two years of Army service), he worked on early prototypes of video tape recorder technologies for Alexander M. Poniatoff and Charlie Ginsburg. As a non degree-holding "consultant", Dolby played a key role in the effort that led Ampex to unveil their prototype Quadruplex videotape recorder in April 1956 which soon entered production.

In 1957, Dolby received his B.S. in electrical engineering from Stanford. He subsequently won a Marshall Scholarship for a Ph.D (1961) in physics from the University of Cambridge, where he was a Research Fellow at Pembroke College.

After Cambridge, Dolby acted as a technical advisor to the United Nations in India, until 1965 when he returned to England, where he founded Dolby Laboratories in London with a staff of four. In that same year, 1965, he officially invented the Dolby Sound System, a form of audio signal processing. His first U.S. patent was not filed until 1969, four years later. The filter was first used by Decca Records in the UK.

Dolby died of leukemia on September 12, 2013, at his home in San Francisco at the age of 80. Dolby was survived by his wife Dagmar, two sons, Tom and David, and four grandchildren. Kevin Yeaman, President and chief executive of Dolby Laboratories, said "Today we lost a friend, mentor and true visionary." Neil Portnow, President of the National Academy of Recording Arts and Sciences, said Dolby had "changed the way we Listen to music and movies for nearly 50 years" and that Dolby's "technologies have become an essential part of the creative process for recording artists and filmmakers, ensuring his remarkable legacy for generations to come."

In his will, Dolby bequeathed more than $52 million to Pembroke College at the University of Cambridge, the largest single donation received by the University's current fundraising campaign. In December 2017 it was announced that his family has donated a further £85m from his estate to Cambridge University's Cavendish Laboratory.