Carroll was born Janet Carol Thiese in Chicago, the daughter of Hilda Catherine (née Patton) and George Nicholas Thiese. She received formal theatrical training and began acting professionally in the late 1960s, appearing in numerous productions in local theaters. She then became a regular at Starlight Theatre in Kansas City, Missouri, where she acted during five seasons.

Since 1982, she had performed as a singer at Jazz Festivals throughout the United States and Canada, being accompanied by her seven piece format, while interpreting traditional jazz, swing, blues, and classic ballads or the Great American Songbook. She performed in Victoria and Vancouver summer festivals in British Columbia, as well as in Monterey, Los Angeles, Newport Beach, Santa Catalina Island, and New Orleans stages, along with concerts at United Service Organizations shows, the Palmer House in Chicago, the Fountaineblue in Miami Beach, the Ritz-Carlton chain, the Biltmore Hotel in Coral Gables and the San Antonio River Walk (San Antonio, Texas). At the Redding Jazz Festival, she was honored with an award for Best Vocalist. In 2004, she was the featured performer at the Porrath Foundation for Cancer Patient Advocacy Event tribute to film star Rhonda Fleming.

Carroll's career spanned more than four decades and included major roles in Broadway musicals and Hollywood productions, but was perhaps most recognized for her portrayal of the oblivious mother of Joel (Tom Cruise) in the 1983 film Risky Business.

Besides Risky Business, Carroll appeared in more than 20 other films over the next three decades, including Secret Admirer (1985), The Killing Time (1987), Memories of Me (1988), Family Business (1989), Talent for the Game (1991), Destiny Turns on the Radio (1995), Forces of Nature (1999), The Omega Code and Enough (2002), among others. She developed her television career with recurring roles on the series Hill Street Blues (1983, 1986 as Peggy LaRue Nelson), The Bronx Zoo (1987 as Carol Danzig), Murphy Brown (1990–96, as Doris Dial, stoic anchorman Charles Kimbrough's wife), Melrose Place (1993–97, as Marion Shaw, Kimberly's mother), and Still Standing (2002–04, as Helen Michaels).

In 1992, Carroll collaborated as a singer on the album This Joint Is Jumpin' Live! – Beverly Hills Unlisted Jazz Band, a recording project led by dixieland trombonist and actor Conrad Janis. She later released her solo albums Presenting... Janet Carroll and the Hollywood Jazz Cats (1992), I Can't Give You Anything But Love (2000), I'll Be Seeing You (2000) and Lady Be Good (2010).

From 2004-05, she starred on Broadway creating the role of "Aunt March" in the original musical Little Women, which is based in the 1869 novel of the same title written by American author Louisa May Alcott. She promoted brands such as Century 21, Diet Coke, Outback Steakhouse and Holiday Inn, among others, in television advertisement spots.

By 2011, she was preparing the production of her fourth and fifth records titled A Tribute to the Great Ladies of Song! and Scorch Your Shorts Torch Songs!. She was diagnosed with brain cancer later that year and took a leave of absence. She underwent surgery and chemotherapy without success.

Carroll died on May 22, 2012, from brain cancer at her home in Manhattan, aged 71. She was predeceased by one of her sons, Tom Brown. She was survived by another son, George Brown.

Carroll then performed in Kansas City and Chicago, assuming significant roles in such musicals as Carousel, Guys and Dolls, Gypsy, Hello, Dolly!, Mame, South Pacific and The Pajama Game, before moving to California, where she continued her stage work, winning a Drama-Logue Award for her performance as Klytemnestra in Ezra Pound’s Elektra.