Taylor was born in New York, New York. His mother was Jewish and his father was a "WASP" from Tennessee. Taylor played bass guitar in The Gamblers, one of the first rock groups to play instrumental surf music. Its personnel also included Elliot Ingber, a Future member of Frank Zappa's Mothers of Invention, Fraternity of Man and Captain Beefheart's The Magic Band; Bruce Johnston, half of the Bruce and Terry duo with Terry Melcher from 1962–66 and longtime "sixth" member of The Beach Boys, for a time brother Mel Taylor, and guitarist-songwriter-bandleader Derry Weaver, who would record and perform in several capacities during the early 1960s. The Gamblers had a local hit in the Los Angeles area with "Moon Dawg" and Taylor played on the recording.

Taylor played with Canned Heat from 1967 to 1970, and appeared with them at various festivals including the Monterey International Pop Festival and Woodstock .

His band nickname was "The Mole." In addition to playing bass, he also played lead guitar on occasion. An Example can be heard on the track "Down In the Gutter, But Free," on the album Hallelujah. In 1969, due to a dispute with Taylor, Henry Vestine left the band. Guitarist Harvey Mandel filled the void as the band's lead Guitarist. In 1970, when John Mayall moved to Los Angeles, Taylor and Mandel quit Canned Heat to join him in the Bluesbreakers. After the Bluesbreakers tours, Taylor played briefly with the Sugarcane Harris Band (later called Pure Food and Drug Act).

In 1974, Taylor became part of The Hollywood Fats Band led by Mike "Hollywood Fats" Mann. The two of them joined Canned Heat for a King Biscuit Flower Hour concert in 1979. Taylor recorded Reheated in 1988, again with Canned Heat. He toured and recorded with his former band a few more times until 1999. In 2007, Taylor and Harvey Mandel reunited with Fito de la Parra and the rest of the current Canned Heat line-up to perform certain shows. Taylor, Mandel and de la Parra were all in the line-up that played Woodstock. The three members of Canned Heats Woodstock line-up have toured extensively from 2009 to 2013.

Recently, Taylor has become a leading exponent and practitioner of the acoustic upright bass in the contemporary blues scene. He is quite prominently seen with his upright bass in the live blues film, Lightning in a Bottle. He was also featured in a concert DVD released in winter 2013, from the album Time Brings About A Change by Floyd Dixon. This concert features three elder piano players — Dixon, Pinetop Perkins and Henry Gray — and was filmed at the Rhythm Room in Phoenix, Arizona on June 1 and 2, 2006.

In 2014, Taylor was nominated for a Blues Music Award in the 'Best Instrumentalist – Bass' category.