Johansen began his career in the late 1960s as a lead singer in the local Staten Island band the Vagabond Missionaries and later in the early 1970s as the singer/songwriter in the protopunk band the New York Dolls. The New York Dolls released two albums, the eponymous New York Dolls (1973), and Too Much Too Soon (1974). The bulk of the material was written by Johansen and Guitarist Johnny Thunders. The Dolls were well received critically, but failed to succeed commercially.

During a career that has seen many changes, Johansen has worked consistently with certain Musicians, including Sylvain Sylvain, Drummer Tony Machine (formerly an agent who worked for Leber & Krebs, a member of the New York Dolls from 1975 until 1977, and a fixture in many David Johansen groups and throughout the Buster Poindexter period) and Brian Koonin, Guitarist and banjo player with Buster Poindexter and The Harry Smiths, as well as keyboard player with the New York Dolls for the first reunion gig and the One Day It Will Please Us To Remember Even This CD and tour.

Johansen acted in several films during the 1980s and 1990s, and in 2000, had a brief role on the HBO drama Oz. He appeared in the television series The Adventures of Pete & Pete in the episode "On Golden Pete", in which he played a park ranger. He appeared in the Muppet Television segment of an episode of The Jim Henson Hour. Among his more prominent roles are that of the wisecracking Ghost of Christmas Past in Scrooged as well as the part of "Looney" in the movie Let It Ride, playing opposite Richard Dreyfuss. He starred opposite Hulk Hogan and Sherman Hemsley in Mr. Nanny, and co-starred with John C. McGinley in the movie Car 54, Where Are You?, based on the television series. He can be seen in the Jim White documentary film Searching for the Wrong Eyed Jesus in which he sings a version of Geeshie Wiley's "Last Kind Words" while in a motel room. He also had a supporting role with Mick Jagger and Emilio Estevez in the movie Freejack. He also played Halston (a hired hitman) in the anthology film Tales from the Darkside: The Movie (1990), in the segment "The Cat From Hell". He was also the guest music Artist on the Miami Vice episode "The Dutch Oven" (1985) where he sang "King of Babylon". Johansen most recently portrayed the bartender in the Netflix special A Very Murray Christmas.

In addition to his own albums, Johansen contributed songs to the soundtracks of the films Times Square and The Aviator ("Flowers of the City" and "Ain't Cha Glad" respectively) and guests on About Them Shoes, a CD by veteran blues man Hubert Sumlin. Another non-album track of his, "Johnsonius", appears on the 1984 compilation A Diamond Hidden in the Mouth of a Corpse and "The Rope (The Let Go Song)", a track originally recorded during the sessions for his eponymous first album, and published on the B-side of the single, "Funky But Chic", a song that was performed by the original New York Dolls before breaking up.

In 2004 Johansen reunited with Sylvain Sylvain and Arthur Kane, of the New York Dolls. Owing to the success of the tour, in 2006 the New York Dolls released One Day It Will Please Us to Remember Even This, their first album in nearly thirty years. It was critic Robert Christgau's choice for album of the year. In 2009 the band released Cause I Sez So and in 2011 Dancing Backward in High Heels.

An Artist noted for his musical unpredictability and versatility, Johansen has been a consistent blues enthusiast since the earliest days of the Dolls, with covers of songs by Bo Diddley and Sonny Boy Williamson among their earliest numbers. The 2006 Dolls CD Private World : The Complete Early Studio Demos 1972/3 featured the Dolls performing songs by Otis Redding, Gary US Bonds, Chuck Berry, The Shangri-Las, and Muddy Waters, in addition to versions of songs from their two Mercury albums. Also featured on the CD was a previously unreleased Dolls number, "Endless Party".

In October 2007, Johansen participated in "The Staten Island Composers Project," featuring work by three Musicians who call the island home: Johansen; Vernon Reid, founder of the '80s rock-metal pioneers Living Colour; and Galt MacDermot, best known as the Composer of the musical Hair. The Council on the Arts and Humanities for Staten Island commissioned the program and asked each Artist to write 20 minutes of music conveying something of his connection to the island often referred to as New York City's forgotten borough. Johansen's opus, a cinematic and unabashedly romantic Adagio scored mostly for strings, is called "Mara Dreams the Moon Gate of Uncommon Beauty." Inspired by The Moon Gate of Uncommon Beauty, a round portal between two rockscapes in the Chinese Scholar's Garden at the Staten Island Botanical Garden. In September 2009, he appeared on Anthony Bourdain: No Reservations, the Travel Channel TV program, in which he toured Staten Island with the program's host.