Spinell was born Joseph J. Spagnuolo (Italian pronunciation: [spaˈɲɲwɔlo]) in Manhattan, New York, of Italian immigrant parents, and the last of six children. His father, Pelegrino Spagnuolo (1892–1950), died from liver and kidney disease. His mother, Filomena Spagnuolo (1903–1987), was a bit-part Actress who acted in a few movies, some of them alongside her son. Spinell stood 5 foot and 11 inches. He was born at his family's apartment on Second Avenue in Manhattan's Kip's Bay, an area then home to 10,000 Italian Americans. A few years after the death of his father, he moved with his mother and older siblings to Woodside, Queens, New York where he lived off-and-on for the remainder of his life. He was known to abuse drugs and alcohol heavily on and off throughout his career, especially during periods of unemployment. Spinell also suffered most of his life from hemophilia as well as chronic asthma.

Spinell was married to adult film star Jean Jennings (1957-2011) from February 1977 to July 1979. Together they had one daughter, but they eventually divorced (this was referenced for Spinell's role in the 1980 film Cruising, where his brutal, closeted NYPD patrolman talks about how his wife had abandoned him by taking their child and leaving for Florida, which is very similar to what actually happened to Spinell right before he began filming this role). A close friend of Sylvester Stallone, Spinell was the godfather of his late son Sage Stallone. He was the distant cousin of New York Giants assistant defensive coach Steve Spagnuolo.

Because of his large, heavyset frame and imposing looks, Spinell was often cast as Criminals, thugs, or corrupt police officers. As a teenager and young adult, Spinell starred in various stage plays, both on and off Broadway. His most notable film roles were as mafioso Willie Cicci in The Godfather and The Godfather Part II, and as loan shark Tony Gazzo in Rocky and Rocky II. Although primarily known as a character actor, Spinell co-wrote and starred as a serial killer in the 1980 film, Maniac.

Maniac 2: Mr. Robbie was a 1986 horror short promotional film shot by Joe Spinell to raise financing for a sequel to Spinell's 1980 horror film Maniac. The short film was included with the 30th anniversary edition release of Maniac. The feature-length version of the film was never shot after Spinell's death in 1989.

Joe Spinell died in his apartment located off of Greenpoint Avenue in Sunnyside, Queens, New York on January 13, 1989 at the age of 52. Sometime during that morning, he cut himself badly on his glass shower stall door after apparently slipping in his bathtub while showering and soon afterward fell asleep on his couch instead of calling for help, his hemophilia causing him to bleed to death. His body was found by his friend Sal Sirchia. Sirchia had phoned Spinell that morning and was told of the injury and his refusal of treatment. Sirchia became concerned when he tried calling Spinell from his workplace at around noon and there was no answer. In the late afternoon, Sirchia left work at the end of his shift and drove to Spinell's apartment. After no one answered the front door, Sirchia called the building superintendent, who entered Spinell's apartment, where they both found Spinell dead on his living room couch. Wearing only a bathrobe, Spinell bled to death over part of the couch from an apparent deep cut on the back of his head.