In the 1960s and early 1970s, he appeared in two productions of the New York Shakespeare Festival — Coriolanus at Central Park's Delacorte Theatre (1965), and Václav Havel's The Memorandum — and the David Newbburge-Jacques Urbont musical Stag Movie (1971), in which stars Sullivan, as Rip Cord, and Adrienne Barbeau, as Cookie Kovac, were "quite jolly and deserve to be congratulated on the lack of embarrassment they show when, on occasion, they have to wander around stark naked. They may not be sexy but they certainly keep cheerful", wrote The New York Times theater critic Clive Barnes in an otherwise negative review.

Born in Chicago, Illinois, Sullivan served in the Korean War and then attended the University of Maine. After touring with a stage company, he moved to New York City and studied at the American Theatre Wing. He made his Off-Broadway debut in Red Roses for Me in 1961, and went on to appear in the London company of the musical South Pacific.

In 1972, he made his feature film debut in the military drama Parades (1972; re-released as The Line, 1980). This was followed by an appearance in a CBS TV-movie adaptation of David Rabe Sticks and Bones, a black comedy about a Vietnam War veteran. The subject matter proved so controversial that half of the network's affiliates refused to broadcast the telefilm.

Sullivan was then featured prominently in Director George Roy Hill's hit The Sting (1973), playing Cole, the hired killer who dogs the Robert Redford and Paul Newman characters. Following roles in other productions, Sullivan reteamed with star Newman and Director Hill for Slap Shot (1977), a hit comedy about a down-and-out hockey team. In a departure from the stoic, taciturn parts in which he was often cast, Sullivan played a spectacularly vulgar hockey player, Morris "Mo" Wanchuk.

Sullivan's other feature film credits include Walk Proud (1979), The Island (1980); Ghost Story (1981); Tin Men (1987); The Untouchables (1987); Funny Farm (1988); Dead Bang (1989); The Dream Team (1989); The Abyss (1989); Guilty by Suspicion (1991); True Colors (1991), The Prince of Tides (1991); Sister Act 2: Back in the Habit (1993); The Fantasticks (made 1995, released 2000); The Jerky Boys: The Movie (1995); Canadian Bacon (1995); and Bushwhacked (1995). Of his role as a harsh husband in The Prince of Tides, in which his unwary character is given dog food to eat and consumes it with gusto, Sullivan told an interviewer he was never quite sure if the contents of a can served him by Kate Nelligan, who played his wife, was actually dog food. He added, however, that as an actor he did not believe in questioning a Director, and that whatever it was tasted fine.

Other theater work includes the Off-Broadway plays The Ballad of Soapy Smith by Michael Weller (1984) and Neal Bell's Cold Sweat (1988).

Sullivan retired in 2000, and lived on the Upper West Side of Manhattan. He died on December 31, 2008, aged 77, of cancer.

He followed this with his Broadway debut, playing three different military officers in a revival of David Rabe's play The Basic Training of Pavlo Hummel (April–September 1977), starring Al Pacino. The following year, Sullivan earned a Drama Desk Award nomination for Outstanding Featured Actor in a Musical for his performance as steelworker Mike LeFevre in Working (May–June 1978), adapted from the book by Studs Terkel and also starring Patti LuPone and Joe Mantegna. He would go on to do four other Broadway plays: Beth Henley's The Wake of Jamey Foster (October 1982), with Holly Hunter; a Circle in the Square revival of The Caine Mutiny Court Martial (May–November 1983); Peter Hall's revival of Tennessee Williams's Orpheus Descending (September–December 1989), as Jabe Torrance opposite Vanessa Redgrave's Lady Torrance (both recreating their roles in the TNT cable network's adaptation); and a stage version of the movie On the Waterfront (May 1995).