Gazzara received acclaim for his off-Broadway performance in End as a Man in 1953. The production was transferred to Broadway and run until 1954.

In 1954, Gazzara (having modified his original surname from "Gazzarra") made several appearances on NBC's legal drama Justice, based on case studies from the Legal Aid Society of New York.

Gazzara became a Broadway sensation when he created the role of Brick in Tennessee Williams' Cat On A Hot Tin Roof (1955-56) opposite Barbara Bel Geddes, directed by Elia Kazan, although he lost out to Paul Newman when the film version was cast. He followed it with another long run in A Hatful of Rain (1956)

He joined other Actors Studio members in the 1957 film The Strange One produced by Sam Spiegel. He had a Broadway flop with The Night Circus (1958).

Then came a high-profile performance as a soldier on trial for avenging his wife's rape in Otto Preminger's courtroom drama Anatomy of a Murder (1959).

He went to Italy to make a comedy The Passionate Thief (1960) with Anna Magnani and Totò. Back in the US he was second billed in The Young Doctors (1961) and starred in Convicts 4 (1962). He returned to Italy to make The Captive City (1962) with David Niven.

Gazzara was married three times; First to Actress Louise Erickson (1951–57). He married Actress Janice Rule on November 25, 1961 in San Francisco. They had a daughter named Elizabeth. He married model Elke Krivat in 1982 and remained married to her until his death. Gazzara adopted his wife's daughter Danja from her prior relationship. Following his separation from his first wife, Gazzara was engaged to stage Actress Elaine Stritch and later disclosed a love affair with Actress Audrey Hepburn. He and Hepburn co-starred in two of her final films, Bloodline (1979) and They All Laughed (1981).

Gazzara became well known in several television series, beginning with Arrest and Trial, which ran from 1963 to 1964 on ABC. He also appeared in the TV special A Carol for Another Christmas (1964) and had a short Broadway run in A Traveller without Luggage in 1964.

Gazzara was the male lead in A Rage to Live (1965) with Suzanne Pleshette. He gained fame in the TV series Run for Your Life which ran from 1965 to 1968 on NBC, in which he played a terminally ill man trying to get the most out of the last two years of his life. For his work in the series, Gazzara received two Emmy nominations for "Outstanding Lead Actor in a Drama Series" and three Golden Globe nominations for "Best Performance by an Actor in a Television Series - Drama."

In 1968, during filming of the war movie The Bridge at Remagen, co-starring Gazzara and friend Robert Vaughn, the Soviet Union and its allies invaded Czechoslovakia. The cast and crew were detained for a time; filming was later completed in West Germany. During their departure from Czechoslovakia, Gazzara and Vaughn assisted with the escape of a Czech waitress whom they had befriended. They smuggled her to Austria in a car waved through a border crossing that had not yet been taken over by the Soviet army in its crackdown on the Prague Spring.

When the series ended Gazzara had a cameo in If It's Tuesday, This Must Be Belgium (1969) and a lead in the wartime action film The Bridge at Remagen (1969).

Some of the actor's most formidable characters were those he created with his friend John Cassavetes in the 1970s. They collaborated for the first time on Cassavetes's film Husbands (1970), in which he appeared alongside Peter Falk and Cassavetes himself.

Gazzara starred in a TV movie, Pursuit (1972), the directorial debut of Michael Crichton. He made The Sicilian Connection (1972) in Italy, and did a science fiction film The Neptune Factor (1973).

In addition to acting, Gazzara worked as an occasional television director; his credits include the Columbo episodes A Friend in Deed (1974) and Troubled Waters (1975). Gazzara was nominated three times for the Tony Award for Best Performance by a Leading Actor in a Play—in 1956 for A Hatful of Rain, in 1975 for the paired short plays Hughie and Duet, and in 1977 for a revival of Who's Afraid of Virginia Woolf?, opposite Colleen Dewhurst.

Gazzara appeared on Broadway in Hughie (1975) then worked again for Cassavetes as Director in The Killing of a Chinese Bookie (1976), in which Gazzara took the leading role of the hapless strip-joint owner, Cosmo Vitelli. He starred in an action movie, High Velocity (1976) and was one of many stars in Voyage of the Damned (1976).

Gazzarra returned to Broadway for a production of Who's Afraid of Virginia Woolf? with Colleen Dewhurst in 1976.

A year later, he starred in yet another Cassavetes-directed movie, Opening Night, as stage Director Manny Victor, who struggles with the mentally unstable star of his show, played by Cassavetes's wife Gena Rowlands. He made an acclaimed TV movie The Death of Richie (1977).

Gazzara was the honorary starter of the 1979 Daytona 500, the first flag-to-flag Daytona 500 broadcast live on CBS. He was also featured in a 1994 article in Cigar Aficionado, in which he admitted smoking four packs of cigarettes a day until taking up cigar smoking in the mid-1960s.

Gazzara made some films in Europe: Tales of Ordinary Madness (1981), The Girl from Trieste (1982), A Proper Scandal (1984), My Dearest Son. He starred with Rowlands in the critically acclaimed AIDS-themed TV movie An Early Frost (1985), for which he received his third Emmy nomination.

Gazzara appeared in 38 films, many for television, in the 1990s. He worked with a number of renowned Directors, such as the Coen brothers (The Big Lebowski), Spike Lee (Summer of Sam), David Mamet (The Spanish Prisoner), Walter Hugo Khouri (Forever), Vincent Gallo (Buffalo '66), Todd Solondz (Happiness), John Turturro (Illuminata), and John McTiernan (The Thomas Crown Affair).

Gazzara told Charlie Rose in 1998 that he went from being mainly a stage actor who often would turn up his nose at film roles in the mid-1950s to, much later, a ubiquitous character actor who turned very little down. "When I became hot, so to speak, in the theater, I got a lot of offers," he said. "I won't tell you the pictures I turned down because you'll say, 'You are a fool,' and I was a fool."

In his seventies, Gazzara continued to be active. In 2003, he appeared in Nobody Don't Like Yogi, an off Broadway show about Yogi Berra which had a solid run and was in a revival of Awake and Sing! (2006).

He was in the ensemble cast of the experimental film Dogville, directed by Lars von Trier of Denmark and starring Nicole Kidman, as well as the television film Hysterical Blindness (he received an Emmy Award for his role). In 2005, he played Agostino Casaroli in the television miniseries, Pope John Paul II. He completed filming his scenes in the film The Wait in early 2012, shortly before his death.

Gazzara was diagnosed with throat cancer in 1999. He suffered a stroke in 2005. On February 3, 2012, he died of pancreatic cancer at Bellevue Hospital Center in New York.

Gazzara was born in New York City, the son of Italian immigrants Angelina (née Cusumano) and Antonio Gazzarra, a laborer and carpenter, each of Sicilian origin – Angelina from Castrofilippo and Antonio from Canicattì in the province of Agrigento. Gazzara grew up in New York's Kips Bay neighborhood; he lived on East 29th Street and participated in the drama program at Madison Square Boys and Girls Club located across the street. He attended New York City's Stuyvesant High School, but finally graduated from Saint Simon Stock in the Bronx. Years later, he said that the discovery of his love for acting saved him from a life of crime during his teen years.