Born in New York City in 1932, Rossi earned a Bachelor of Arts degree in communication, arts and theater, Greek and Latin at Loyola Marymount University, Los Angeles and served as an officer in the United States Air Force.

The Comedian was discovered by Mae West, who, with her manager Jerry Franks, saw him in 1953 as the lead of The Student Prince at the Civic Light Orchestra in Los Angeles. Under his birth name (Joseph Tafarella), Rossi was signed to a nightclub show with West at Ciro's in Los Angeles. The show was subsequently moved to the Sahara Hotel in Las Vegas. According to Rossi, West picked "Steve" because she was dating actor Steve Cochran at the time and had a manager by the name of Bernie Ross; an "i" was added to the end of Ross to reflect Rossi's Italian heritage. West and Rossi toured in 1953 and 1954. Rossi wrote the musical numbers he performed in the show with West. Following the tour's end, Rossi concentrated on his musical career, recording music with his group, Steve Rossi and The Robinaires, on Red Robin Records.

In 1957, he formed the Allen & Rossi comedy duo with Marty Allen. Rossi played straight man to Allen, and the pair made many appearances together in the 1950s and 1960s, including 44 appearances on The Ed Sullivan Show. They released multiple comedy albums; co-starred in a feature film, The Last of the Secret Agents?; and also performed live in Las Vegas. The pair was signed in a star studded event at Paramount Studios.

Rossi continued his singing career as a solo Artist, recording for several labels in the 1960s and 1970s, including ABC Records' ABC-Paramount label, Red Bird Records, Roulette Records, and Columbia Records. Milton Berle famously lost a bet of one week's salary to Rossi when Rossi correctly predicted that, at the 1962 Academy Awards, Bob Hope would tell a joke about Elizabeth Taylor. In 1963, Allen & Rossi were the first comedians to go to Washington, D.C., to lobby Congress on the House floor.

Following the breakup of Allen & Rossi in 1968, he teamed for less than three months with Joe E. Ross in an act called "Rossi & Ross". The team of Rossi & Ross played once on Ed Sullivan and disbanded in January 1969. In 1969, he announced a pairing with Comedian Slappy White, creating the comedy act Rossi & White. The interracial comedy team released the album, "I Found Me a White Man—You Find Yourself One" on Roulette Records. In 1971, in an attempt to sell more tickets, Overseas National Airways employed the duo as live onboard entertainment.

In 1975 Rossi was reported by entertainment reporter Earl Wilson to have received a mysterious visitor claiming to be Howard Hughes. Hughes had come to see Rossi in Vancouver to prove he was not dead yet. Rossi later teamed up with Sandy Hackett, son of Buddy Hackett. They appeared at several venues in Las Vegas, including at the Star Bright theater. Rossi and Allen reunited several times for subsequent projects, including a film released in 1974 and stage performances together from 1984–85, and again from 1990-94. Rossi appeared as a guest on The Howard Stern Show many times in the 1980s and early 1990s.

Rossi presented a spoof of Washington, D.C. called "Laffs, Sex & Politics," co-starring Buddy Hackett, in 2000. He had a role in the Brad Pitt movie The Mexican (2001). In 2004, Rossi was inducted into the Show Business Legends Hall of Fame at the Tropicana Hotel in Las Vegas. His book, Adult Stand-up Comedy For Wannabe Comedians: Hilarious Stand-up Routines, Jokes and Stories, was issued in 2006. In 2007, he was part of a touring cabaret show titled "5 Star Revue" with Gary Collins, Mary Ann Mobley, Ruta Lee and Ronnie Schell. In 2009, he appeared along with Schell off-Broadway in Don't Leave it All to Your Children. He also performed his stand-up comedy act at Feinstein's in New York City.

Permanently living in Las Vegas, Nevada, he used his website blog to promote his upcoming appearances. In August 2010, the Comedy Talks speakers series of the University of San Francisco hosted a panel discussion including Rossi, Rich Little, and Carol Channing. In 2012, Rossi re-released "The World's Dirtiest Joke Book" in electronic book format. The same year, he attended the opening of Rich Little's show "Jimmy Stewart and Friends", with strip headliner Terry Fator. Steve is credited as co-writer of the play. Upon the death of his friend Dick Clark in 2012, Rossi appeared in local Las Vegas media to share his memories.

Rossi died in a Las Vegas hospice on June 22, 2014 from cancer, aged 82. He was survived by his wife, Karma, two children and two grandchildren.