Aiello broke into films in the early 1970s. One of his earliest roles came as a ballplayer in the 1973 baseball drama, Bang the Drum Slowly, with Robert De Niro. Aiello had a walk-on role as small-time hood Tony Rosato in The Godfather Part II (1974), ad-libbing the famous line "Michael Corleone says hello!" during a hit on rival gangster Frank Pentangeli (Michael V. Gazzo).

Aiello's Broadway theatre credits include Gemini, The Floating Light Bulb, Hurlyburly, and The House of Blue Leaves. He also was in the 1976 Broadway play Wheelbarrow Closers, directed by Paul Sorvino.

In 1980, Aiello had a co-lead role with Jan Michael Vincent in Defiance, about some Manhattan residents who fight back against the thugs terrorizing the neighborhood. The next year, he received considerable acclaim for playing a racist New York City cop in Fort Apache, The Bronx (1981) with Paul Newman. In 1981, Aiello won a Daytime Emmy Award for his appearance in an ABC Afterschool Special called A Family of Strangers.

He was paired with De Niro again for the Sergio Leone gangster epic, Once Upon a Time in America (1984), as a police chief whose name was also "Aiello." His many film appearances included two for Director Woody Allen, who cast him in The Purple Rose of Cairo (1985) and Radio Days (1987).

He played the title character for the video of Madonna's song, "Papa Don't Preach" (1986), and recorded his own answer song, "Papa Wants the Best for You", written by Artie Schroeck.

Aiello has also portrayed more sympathetic characters. He gained recognition as the befuddled fiancé of Cher opposite her Oscar-winning performance in the romantic comedy Moonstruck (1987), and the actor made a comic appearance in drag for the Robert Altman fashion-industry film Prêt-à-Porter (1994). He also had sympathetic roles in the 1990 horror thriller Jacob's Ladder and the 1991 comedy-drama 29th Street.

Aiello is perhaps best known for his role as pizzeria owner Sal in Spike Lee's Do the Right Thing (1989). At the time of the film's release, in an interview with the Chicago Tribune, he called the role his "first focal part". He further identified the film as a very collaborative effort, during which Spike Lee at one point told him "Whatever you wanna do, you do." Aiello went on to write a crucial scene he shared with John Turturro ten minutes prior to its production. The role earned him nominations for a Golden Globe Award and the Academy Award for Best Supporting Actor, as well as film critic awards from Boston, Chicago and Los Angeles.

Aiello's singing has been on display in films such as Hudson Hawk (1991), Once Around (1991), and Remedy that starred his son Ricky Aiello and Jonathan Doscher. He has released several albums featuring a big-band sound, including I Just Wanted to Hear The Words (2004), Live from Atlantic City (2008), and My Christmas Song for You (2010). Aiello and EMI Songwriter Hasan Johnson released an album of standards fused with rap entitled Bridges in 2011.

Aiello played nightclub owner and Lee Harvey Oswald Assassin Jack Ruby in the 1992 biopic Ruby and a political big shot with mob ties in City Hall (1996), starring Al Pacino. He later starred in the independent feature film Dolly Baby (2012), written and directed by Kevin Jordan; Aiello also starred in Jordan's Brooklyn Lobster, which premiered at The Toronto Film Festival in 2005.

Aiello, the fifth of six children, was born on West 68th Street, Manhattan, the son of parents, Frances (née Pietrocova), a seamstress, and Daniel Louis Aiello, a laborer, who deserted his children and wife, who had lost her eyesight and become legally blind. For many years, Aiello publicly condemned his father but the two reconciled in 1993, although Aiello harbors a resentment of his father's conduct. He moved to the South Bronx when he was seven, and later attended James Monroe High School.

He is the father of stuntman and actor Danny Aiello III, who died in 2010 of pancreatic cancer, His surviving children are Rick, Jaime, and Stacey Aiello.

In July 2011, Aiello appeared Off Broadway in the two-act drama The Shoemaker, written by Susan Charlotte and directed by Antony Marsellis. The play is a stage version of his 2006 movie A Broken Sole, which began life in 2001 as a one-act play.

In 2014, Aiello published his autobiography, I Only Know Who I Am When I Am Somebody Else: My Life on the Street, On the Stage, and in the Movies via Simon & Schuster.