Andy Karl was born in Baltimore, Maryland. He attended Towson High School, where he was on the football team and performed in theater. In high school, he played Jud Fry in Oklahoma! and General Bullmoose in Li'l Abner. At age 16 he performed as the title role in Aladdin at the White Marsh Dinner Theatre. After high school, he attended Towson University where he studied voice and music. Karl originally planned on becoming a voice Teacher until the Director of a show he was in at college told him he had what it took to be a professional actor. Karl continued to perform in shows while in college. In Winter 1993–1994, he played the 9-foot dragon in the children's musical The Reluctant Dragon at Toby's Dinner Theatre in Columbia, Maryland.

Karl moved to New York in 1994. He was an understudy in the touring cast of The Who's Tommy and in 1998 played Rum Tum Tugger in the touring cast of Cats. In February 2000 Karl was an understudy in the New York premiere of Sondheim's Saturday Night at the Second Stage Theatre. He made his Broadway debut in 2000 as the replacement for Joey in Saturday Night Fever. He continued with the production, this time in the role of Tony Manero, in its first national tour. In 2001 he performed in the musical stage production Me and Mrs. Jones at the Prince Music Theater in Philadelphia.

Karl met his wife Orfeh while in the musical for Saturday Night Fever. They were married in January 2001 and reside in Manhattan. In addition to Saturday Night Fever, Karl and his wife have performed together in Me and Mrs. Jones, Nights on Broadway II, Bright Lights, Big City and Legally Blonde.

Karl was next seen as Dino Del Canto in the world premiere of the musical Romeo and Bernadette at the Coconut Grove Playhouse in Miami, Florida. The show, which ran in January 2003, garnered Karl a Carbonell Award for Best Actor in a musical. He reprised the role at the Paper Mill Playhouse in Milburn, New Jersey in February and March, 2003.

Karl was also seen at the Paper Mill Playhouse in June and July of that year, when he performed as Danny Zuko in Grease. In January 2004 he performed in the musical revue Nights on Broadway II at Caesar's Palace in Atlantic City, New Jersey.

On September 26, 2005, Karl performed in the Actor's Fund Concert production of On the Twentieth Century. Karl's next role was the title character Adam Patterson in the Off-Broadway musical Slut at the American Theatre of Actors in Fall 2005.

In January 2006 Karl was in the three-person cast of Hunka Hunka Burnin' Love, a celebration of the music of Elvis Presley, at the now-closed Au Bar. In June and July of that year, he starred as Tad in the musical Bright Lights, Big City at the Prince Music Theater in Philadelphia. He then joined the Broadway cast of The Wedding Singer at the Al Hirschfeld Theatre, where he played the roles of Drunk Dave and Bad Haircut Guy until the show closed at the end of 2006.

Karl's next role was in Legally Blonde, where he played Kyle "the UPS guy", Dewey, and Grandmaster Chad at the Golden Gate Theatre in San Francisco, where the musical had its pre-Broadway tryout from January 23 to February 24, 2007. He reprised his roles in the Broadway production, which started previews on April 3 and opened on April 29, 2007 at the Palace Theatre. He was also an understudy to Emmett and Professor Callahan. He left the show on July 20, 2008.

Karl briefly left Legally Blonde in June 2007 to work on 9 to 5 in its week-long workshop. After leaving Legally Blonde, Karl continued working on the musical 9 to 5. He played Joe, the love interest of Allison Janney's character, in the pre-Broadway tryout in Los Angeles. The show played at the Ahmanson Theatre from September 3 to October 19, 2008. The show opened on Broadway on April 30, 2009 at the Marquis Theatre, and closed on September 6, 2009.

Karl took over the role of Fiyero from Kevin Kern in the Broadway production of Wicked on February 2, 2010 and played the role until January 2, 2011.

He took over the role Tommy Devito in the Broadway production of Jersey Boys on October 11, 2011.

Karl played Neville Landless in the Broadway revival of The Mystery of Edwin Drood which opened on November 29, 2012 at Studio 54. For this performance, he was nominated for the Drama Desk Award for Outstanding Featured Actor in a Musical.

Karl and Margo Seibert were featured as Rocky Balboa and Adrian in the new Lynn Ahrens and Stephen Flaherty musical Rocky the Musical, which opened on Broadway in February 2014. In order to get into shape to play Rocky Balboa, Karl put on 12 pounds of muscle and worked with a trainer to get to a heavyweight fighter’s weight. Karl received a nomination for the Tony Award for Best Leading Actor in a Musical for this performance.

In 2015, Karl joined the recurring cast of Law & Order: Special Victims Unit as the unit's new sergeant, Mike Dodds, who is the son of Peter Gallagher's Chief william Dodds. The role reunites Karl and Gallagher after working together on On the Twentieth Century. He has since left his role on Law & Order: Special Victims Unit, which resulted in his character's death.

On January 28, 2016, it was announced that Karl will be playing the role of Phil in the musical adaptation of Groundhog Day, the role played in the 1993 movie by Bill Murray. The musical ran at the Old Vic Theatre London from Monday 11 July 2016 - Saturday 17 September 2016. Karl is currently reprising the role in the Broadway production of Groundhog Day, which opened on April 17, 2017. For his performance in Groundhog Day in London, he received the 2017 Olivier Award for Best Actor in a Musical. On August 15, 2017, the producers of Groundhog Day announced it would end its Broadway run on September 17, 2017, but that a national tour and a possible return to London would follow.

He recently starred as Phil Connors in the musical Groundhog Day on Broadway, a role he originated in London having won the 2017 Laurence Olivier Award for Best Actor in a Musical.