O'Connell began his acting career at a young age. As a child, he did commercial work for Duncan Hines cookies. Shortly after turning eleven, he landed his first feature film role, as the character Vern Tessio in Stephen King's Stand by Me. In 1987, O'Connell appeared on a commercial for Frosted Flakes. O'Connell starred in the Canadian science fiction sitcom My Secret Identity from 1988 to 1991 as the teen hero who develops superhuman traits. During a summer break from NYU, Jerry starred in the feature film Calendar Girl alongside Jason Priestley and Gabriel Olds. He also appeared in the short-lived ABC sitcom, Camp Wilder with Jay Mohr and Hilary Swank in 1992. In his Junior year, O'Connell auditioned for the TV pilot Sliders (also filmed in Canada). He was offered the role of Quinn Mallory in the series, which ran for three seasons on Fox and two seasons on the Syfy Channel. He served as Producer during his fourth and final season, and is credited with writing and directing several episodes.

O'Connell was born in Manhattan, the son of Linda, an art Teacher, and Michael O'Connell, an advertising agency art Director. His maternal grandfather, Charles S. Witkowski, was mayor of Bayonne, New Jersey. He is of half Irish, one quarter Italian, and one quarter Polish, ancestry. O'Connell was raised in Manhattan with his younger brother, actor Charlie O'Connell. As a teenager, he attended Manhattan's Professional Children's School. He attended New York University from 1991–95, majoring in film. While there, he studied screen writing and competed on the fencing team, serving a stint as captain of the sabre squad. He graduated in 1995.

O'Connell has since gone on to star in such movies as Jerry Maguire, Body Shots, Mission to Mars, Tomcats, Scream 2, and Kangaroo Jack. O'Connell has also tried his hand at screenwriting and sold his first screenplay, for First Daughter, to New Regency in 1999. The film was released in 2004 by 20th Century Fox-based Davis Entertainment. O'Connell served as executive Producer on the film, which starred Katie Holmes and Michael Keaton.

O'Connell starred as Detective Woody Hoyt on the NBC crime drama Crossing Jordan (2001) until its cancellation in May 2007, and was engaged to Actress Rebecca Romijn. He starred opposite her in the Ugly Betty episode "Derailed". O'Connell played Hoyt in several episodes of Las Vegas. In 2004, he wore a diaper on Last Call with Carson Daly, during a mock commercial skit for the GoodNites bedwetting product. The National Enquirer caught him filming the skit and printed a photo of him in his diaper.

On July 14, 2007, O'Connell married Actress and former model Rebecca Romijn.

After weeks of officially stating that the couple were "trying" to get pregnant, a publicist for the pair confirmed that Romijn was pregnant with twin girls. When talking about having children, O'Connell said "I am excited I am having girls. I know guys are supposed to say, 'I want a boy. I want to play baseball,' but I think I'm going to be good raising girls...I can't wait. They're going to be tough chicks. They're only having sex when I'm dead. If they don't date boys at all and just want them to be infatuated with their father, that's what I'm really going for." The girls, Dolly Rebecca Rose and Charlie Tamara Tulip O'Connell, were born on December 28, 2008. Dolly is named for singer Dolly Parton and Charlie is named for O'Connell's brother.

O'Connell enrolled in Southwestern Law School in Los Angeles in August 2009 but dropped out to return to acting full-time.

O'Connell appeared on Broadway at the Golden Theatre in Seminar, written by Theresa Rebeck. The play, directed by Sam Gold, opened on November 20, 2011, starring Alan Rickman, Lily Rabe, Hamish Linklater and Hettienne Park. On June 4, 2012, it was announced that O'Connell would play Herman Munster in a reboot of The Munsters, titled Mockingbird Lane; the remake series was never picked up for production. On May 15, 2013, it was announced that O'Connell would play Stuart Strickland on We Are Men, alongside Tony Shalhoub.

In April 2015, O'Connell returned to Broadway as a ghostwriter in the comedy Living on Love, with Renee Fleming, Anna Chlumsky and Douglas Sills. It closed after 37 performances.