As a child, she began singing on a local gospel radio show and was also a member of the church choir. At the age of 15, she began performing with the Renaissance Ensemble that played at area coffee houses and gay bars. On July 5, 1965, Hardy, then 16 years old, was raped at gunpoint by a man she knew who gave her a ride home from a performance with the Renaissance Ensemble. Hardy became pregnant and gave birth to a daughter, Tracey, the following year. Hardy attempted to raise Tracey alone, but found it too difficult. She sent Tracey to live with her elder sister Willie (Carter would later claim Tracey was the product of a short lived marriage, but revealed the truth in an interview in 1994).

Beginning her career in 1970, Carter started in theater; singing and later crossed over to television. Carter was perhaps best known for her role as Nell Harper on the NBC sitcom Gimme a Break! which originally aired from 1981 to 1987. Carter received two Emmy and two Golden Globe award nominations for her work on the series. Prior to Gimme a Break!, Carter won a Tony Award for Best Performance by a Featured Actress in a Musical in 1978 for her performance in the Broadway musical Ain't Misbehavin', as well as an Primetime Emmy Award for her reprisal of the role on television in 1982.

Carter made her Broadway debut in the 1971 rock opera Soon, which closed after three performances. She was the Music Director for the 1974 Westbeth Playwrights Feminist Collective's production of "What Time of Night It Is". Carter appeared alongside Bette Davis in the 1974 stage musical Miss Moffat, based on Davis' earlier film The Corn Is Green. The show closed before making it to Broadway. She broke into stardom in the musical Ain't Misbehavin, for which she won a Tony Award in 1978. She won an Emmy for the same role in a televised performance in 1982.

In 1978, Carter was cast as Effie White in the Broadway musical Dreamgirls, but departed the production during development to take a television role on the ABC soap opera, Ryan's Hope in New York. When Dreamgirls premiered in late 1981, Jennifer Holliday had taken over the lead. In 1981, Carter also took a role on television's The Misadventures of Sheriff Lobo, before landing the lead role of Nell Harper on the sitcom Gimme a Break!.

Additional Broadway credits included Dude and Annie. In 1979, she had a part in the Miloš Forman-directed musical film adaptation of Hair. Her vocal talents are showcased throughout the motion picture Soundtrack.

Carter self-identified as a bisexual. After Gimme a Break! began, Carter's life took a turbulent turn. She attempted suicide in the early 1980s, and entered a drug detoxification facility around 1985. Her brother, Bernard, died of complications due to AIDS in 1989. Carter married Mathematician and lumber executive George Krynicki, and converted to Judaism in 1982 (she had been born into a Roman Catholic family and raised Presbyterian).

The series was a ratings hit for NBC and earned Carter a Golden Globe and Emmy Award nominations. Gimme a Break! aired from 1981 to 1987. In August 1987, after the cancellation of Gimme a Break!, Carter returned to the nightclub circuit with a five-month national tour with Comedian Joan Rivers. In 1989, she shot a pilot for NBC entitled Morton's By the Bay, which aired as a one-time special in May of that year. In this, Carter played the assistant to the owner of a banquet hall, and the focus was on her and her mad-cap staff. Alan Ruck and Jann Karam co-starred. NBC passed on the series development. In October of that same year, she performed "The Star-Spangled Banner" prior to Game 4 of the 1989 World Series, played at Candlestick Park in San Francisco, California.

Carter filed for divorce from Krynicki in 1989; which was finalized in 1992. Carter had three children: two daughters Tracey and Tiffany son Daniel. She adopted both Tiffany and Daniel as newborns over a four-month period. She attempted to adopt twice more but both adoptions fell through. In her first attempt, she allowed a young pregnant woman to move into her home with the plan that she would adopt the child, but the mother decided to keep her baby. In 1992, Carter had surgery to repair two aneurysms and married Roger Larocque in June of that year, later divorcing Larocque the next year. Carter declared bankruptcy in 1995 and again in 2002. She also endured three miscarriages.

In the mid-1990s, Carter appeared on Broadway in a revival of Annie as Miss Hannigan. She was upset when commercials promoting the show used a different Actress, Marcia Lewis, a white Actress, as Miss Hannigan. The producers stated that the commercials, which were made during an earlier production, were too costly to reshoot. Carter said racism played a part in the decision. "Maybe they don't want audiences to know Nell Carter is black", she told the New York Post. "It hurts a lot", Carter told the Post, "I've asked them nicely to stop it—it's insulting to me as a black woman." Carter was later replaced by Sally Struthers.

In 2001, she appeared as a special guest star on the pilot episode of the new WB show Reba and continued with the show, making three appearances in season one. The following year, Carter made two appearances on Ally McBeal.

On January 23, 2003, Carter collapsed and died at her home in Beverly Hills. Her body was discovered by her son, Joshua. Per a provision in Carter's will, no autopsy was performed. Using blood tests, X-rays and a physical examination, the Los Angeles County ruled that Carter's death was the result of "probable arteriosclerotic heart disease, with diabetes a contributing condition." She is survived by her three children. Carter is buried at Hillside Memorial Park Cemetery, Los Angeles.

The following year had her rehearsing for a production of Raisin, a stage musical of A Raisin in the Sun in Long Beach, California, and filming a movie, Swing. Carter's final onscreen appearance was in the comedy film Back by Midnight. It was released in 2005, two years after her death.