Born in Brooklyn, New York City, Perry came to his interest in rock music young. In 1955, at the age of 12, he attended the first of Alan Freed's live rock shows at the Paramount Theatre. He began his career in rock music as a local performer during his adolescence. He attended Poly Prep Country Day School and graduated in 1960. After graduating from the University of Michigan, he shifted into songwriting briefly, collaborating with Kenny Vance. Perry was launched on his career as a Producer, with early projects including Captain Beefheart's debut Safe as Milk and Fats Domino's Fats Is Back. In 1967, Perry moved to Los Angeles and in 1968 produced God Bless Tiny Tim, the debut album of Reprise Records Artist Tiny Tim. The album was Perry's first charting hit, reaching #7 on Billboard magazine's Pop Albums chart.

According to Allmusic's Bruce Eder, the 1970s found Perry "the most renowned Producer in the field of popular music"; Eder goes on to indicate that "his mere involvement with a recording project was enough to engender a mention in the music trade papers and even the popular music press, and the array of gold- and platinum-selling albums with which he was associated made his name synonymous with success." As early as 1973, Village Voice said of Perry that "the rungs on the ladder of success seem so much closer together when Perry is your guide."

In 1978, Perry launched his own label, Planet Records, which he ran for six years until its 1983 sale to RCA, by which point Perry had produced throughout his career at least fifteen gold records (four of which had gone platinum) and a dozen gold singles. Perry had achieved these figures by February 2, 1982. Among the label's roster during his tenure were acts such as Billy Thermal, Bates Motel, the Plimsouls, The Cretones, Bill Medley and the Pointer Sisters, whose charting album Energy was the label's debut. After leaving Planet Records, Perry continued producing some of its acts, including the Pointer Sisters, as well as producing efforts by such artists as Streisand, Donna Summer, Julio Iglesias, Neil Diamond, and Randy Travis. While pursuing these projects, Perry spent the latter part of the 1980s also pulling together a passion project, 1989's Rock, Rhythm & Blues, which featured contemporary artists like Elton John, Rick James, and Chaka Khan performing classic rock songs by Musicians of the 1950s and early 1960s. Also in 1978, Perry played the part of a record Producer in the film American Hot Wax, which was based on the life of disc jockey Alan Freed.

In the 1990s and the 2000s, he worked with Ray Charles on 1993's My World, which was a minor chart success, reaching #145 on Billboard 200. He is credited with helping to craft Rod Stewart's charting pop standards albums in the Great American Songbook series, including It Had to Be You: The Great American Songbook. Perry would go on to co-produce the first three records in the series. In 2004, he reunited with Carly Simon, with whom he had worked earlier in his career. The resultant collaboration was 2005's Moonlight Serenade, which reached #7 on the Billboard 200 and was also a top Internet download. In 2006, he re-entered the studio with another previous collaborator, Art Garfunkel, receiving both Producer and singing credits on 2007's Some Enchanted Evening.

Perry and Actress Jane Fonda were in a relationship, which began in 2009 and ended in 2017.