Williams worked for a pharmaceutical company. However, inspired by Janet Jackson's Rhythm Nation 1814, he left school and quit his job, against the wishes of his family, to pursue a career as a Dancer. During a year in which he was intermittently homeless, Williams visited record labels and dance studios looking for work. He got a job as a background Dancer on a music tour for Kym Sims' dance anthem Too Blind To See It, which led to more work appearing as a Dancer in videos and on tours, such as with George Michael, Madonna, as well as some modeling work. He also choreographed Crystal Waters' 1994 single "100% Pure Love".

Williams is well known for a large facial scar received during a bar fight on Jamaica Avenue, New York City, on his 25th birthday, in which he was slashed with a razor blade. The scar became his signature feature, and resulted in offers to perform as a thug in music videos and modeling opportunities with noted Photographers like David LaChapelle.

He was given one of his first acting roles after being discovered by Tupac Shakur and was cast as High Top, the brother and henchman to Shakur's drug kingpin Tank, in the 1996 film Bullet.

Williams is known for his portrayal of Omar Little in The Wire, which began filming in 2002. The character was based on Donnie Andrews, as well as other crime figures in Baltimore. Williams received the part after only a single audition. He was initially told that the character was slated to appear in just seven episodes of the first season and feared that the character would be killed before the end of the season. However, creator David Simon stated that they always planned to keep the character as part of the continuing ensemble should the show be renewed beyond the first season.

During his portrayal of Omar Little, Williams went by his character's name and developed a habit of smoking cannabis and an addiction to cocaine in 2004.

Williams makes a brief appearance as the shooter at the beginning of the music video for Young Jeezy's "Bury Me a G". He appears in The Kill Point as recurring guest star Q, a police sniper, alongside The Wire co-stars J. D. Williams, Michael Hyatt and Leo Fitzpatrick. He auditioned for the starring role of Mr. Cat but was forced to take a smaller role due to scheduling conflicts; the part of Mr. Cat went to J. D. Williams instead. Williams played a Boston area detective named Devin Amronklin in the 2007 film Gone, Baby, Gone. The film is based on a novel by Dennis Lehane, who has written for The Wire, and was adapted and directed by Ben Affleck. Amronklin is a recurring character in Lehane's Kenzie-Genarro series of books. Williams says that he enjoyed working with Affleck and characterized him as a passionate and hands-on Director. The film also featured his co-star from The Wire, Amy Ryan.

In 2008, then-U.S. Senator Barack Obama cited The Wire as his favorite television show, and called Omar his favorite character. About Omar, Obama said, "That’s not an endorsement. He's not my favorite person, but he's a fascinating character... he's the toughest, baddest guy on the show."

Williams also starred in the film A Day in the Life, which was directed, produced and stars Rapper Sticky Fingaz. The entire film is a musical with every line being delivered in rap verse. Williams starred in HBO's Boardwalk Empire for its five seasons (2010–2014). He appeared as Albert "Chalky" White, the leader of 1920s' Atlantic City's black community.

In November 2011, it was announced that Williams would appear in Quentin Tarantino's feature film Django Unchained. Williams, who had previously confirmed that he was actually in talks with Tarantino to take on the titular role of Django, was to portray a minor character in the film, but scheduling conflicts with Boardwalk Empire prevented him from doing so.

On May 16, 2012, Williams announced that he was an executive Producer of the independent film Snow on tha Bluff, Williams' first film under his company, Freedome Productions. On Power 105.1fm's The Breakfast Club, Williams revealed the June 19 release date for Snow on tha Bluff, describing the movie as "real graphic": "everything that is wrong with the 'hood is in this movie". Williams also shared on The Breakfast Club that he was starring in an African American western, They Die by Dawn, with his co-star Snoop from the HBO series The Wire. Williams also revealed that he was starring in the lead role as Rapper Ol' Dirty Bastard (ODB) from the Wu-Tang Clan in the movie Dirty Whiteboy in 2014, which is based on the relationship ODB had with his manager during the last two years of his life. Williams mentioned the role was special to him because he grew up listening to Ol' Dirty Bastard and to Wu-Tang and he is also a Brooklyn native.

In 2013, he starred in MGMT's music video for "Cool Song No. 2".

Williams was featured modeling for The Gap's 2014 fall collection.

Also in 2015 Williams appeared in the music video for "The Mephistopheles of Los Angeles" by Marilyn Manson.

For his portrayal of Omar, Williams was named by USA Today as one of ten reasons they still love television. Omar was praised for his uniqueness in the stale landscape of TV crime dramas and for the wit and humor that Williams brings to the portrayal. Omar has been named as one of the first season's richest characters, a Robin Hood of Baltimore's west side projects. The Baltimore City Paper named the character one of their top ten reasons not to cancel the show and called him "arguably the show’s single greatest achievement". In 2007 he was nominated for an NAACP Image Award for Outstanding Actor in a Drama Series for his role as Omar.