He was born in to a Jewish family in Brooklyn, New York, the son of Jean (née Gerson), a piano player for dance classes, and David Mazursky, a laborer. Mazursky's grandfather was an immigrant from Ukraine. Mazursky graduated from Brooklyn College in 1951. He was an atheist.

Mazursky began his film career as an actor in Stanley Kubrick's first feature, Fear and Desire (1953). Kubrick asked for verification of his name for the credits and at that point he decided on a first-name change to Paul. Two years later he appeared in a featured position as one of a classroom of teenagers with issues towards authority in The Blackboard Jungle (1955). His acting career continued for several decades, starting with parts in episodes of television series such as The Twilight Zone and The Rifleman.

Soon after starting his acting career, Mazursky became a Writer and worked on The Danny Kaye Show in 1963. In 1965, he collaborated with Larry Tucker in crafting the script of the original pilot of The Monkees television series, in which they both also appeared in cameos.

Mazursky's debut as a film screenplay Writer was the Peter Sellers comedy I Love You, Alice B. Toklas (1968). The following year he directed his first film Bob & Carol & Ted & Alice (produced and written by Mazursky and Larry Tucker), which proved to be a major critical and commercial success. The film was the fifth highest grossing of the year and earned Mazursky his first Oscar nomination.

Mazursky received five Academy Award nominations, four for his screenplay writing on Bob & Carol & Ted & Alice (1969), Harry and Tonto (1974), An Unmarried Woman (1978) and Enemies, a Love Story (1989), and once as Producer of An Unmarried Woman (nominated for Best Picture). He was also twice nominated for a Golden Globe and twice for the Cannes Film Festival's Palm d'Or, among many other awards.

Other films made by Mazursky during this time include the Hollywood satire Alex in Wonderland (1970), the cutting Los Angeles relationship comedy Blume in Love (1973), the semi-autobiographical coming of age story Next Stop, Greenwich Village (1976), the New York City-based Jules and Jim homage Willie & Phil (1980), the contemporary Shakespeare comedy Tempest (1982), the Caribbean-set political farce Moon over Parador (1988), and the acclaimed Isaac Bashevis Singer adaptation Enemies, a Love Story (1989).

Mazursky also played supporting roles in The Other Side of the Wind (1972; finished 2015), A Star Is Born (1976), History of the World Part I (1981), Into the Night (1985), Punchline (1988), Man Trouble (1992), Carlito's Way (1993), Love Affair (1994), 2 Days in the Valley (1996), Miami Rhapsody (1995), Crazy in Alabama (1999), and I Want Someone to Eat Cheese With (2006). He also performed the voice of the Psychologist in Antz (1998).

His career behind the camera continued for the next two decades as he wrote and directed a prolific string of quirky, dramatic and critically popular films. His most successful films were contemporary dramatic comedies and include the Academy Award-winning Harry and Tonto (1974), the Best Picture-nominated An Unmarried Woman (1978), and popular hits such as Moscow on the Hudson (1984) and Down and Out in Beverly Hills (1986). In light of his comedies that tackled a number of modern social subjects, The Hollywood Reporter has stated that "from the late '60s through the '80s, [he] seemed to channel the zeitgeist..." and Variety has stated that "his oeuvre smacks of cultural significance."

Film critic Roger Ebert was a particular fan of Mazursky's work, giving six of his films the optimal four stars in his reviews. In 1986, Ebert stated that "Mazursky has a way of making comedies that are more intelligent and relevant than most of the serious films around."

Late in his life, Mazursky was developing a Broadway musical adaptation of his 1988 film Moon Over Parador.

Mazursky experienced less success in the 1990s, beginning with Scenes from a Mall (1991), starring Woody Allen and Bette Midler. Following his filmmaking satire The Pickle (1993), which was his last writing credit, Mazursky worked only sporadically as a Director on such films as Faithful (1996), Winchell (1998), and Coast to Coast (2003). His final film was the independent documentary Yippee (2006).

In his autobiography Show Me the Magic (1999), Mazursky recounts his experiences in filmmaking and with several well-known screen personalities including Peter Sellers.

In 2000, he was the recipient of the Austin Film Festival's Distinguished Screenwriter Award.

In 2010, the Los Angeles Film Critics Association honored him with an award for Career Achievement.

From 2011 until his death in 2014, Mazursky served as a film critic for Vanity Fair.

On December 13, 2013, Mazursky was awarded the 2,515th star of the Hollywood Walk of Fame, in front of Musso & Frank Grill. Friends and collaborators Mel Brooks, Richard Dreyfuss, and Jeff Garlin were all present.

Paul Mazursky went into cardiopulmonary arrest and died on June 30, 2014, aged 84, at Cedars-Sinai Medical Center in Los Angeles. He is survived by his wife Betsy and his daughter Jill, who works as a writer/producer in Hollywood, as well as grandchildren Carly, Kate, Molly and Tommy, and great-grandson Luca. His daughter Meg died in 2009.

In 2015, Joe Swanberg's film Digging for Fire was dedicated in memory to Mazursky.