Ullman's first professional role was as Louis in the national touring company of The King and I along with Jesse McCartney. Other productions he worked on include Peter Pan at the Polka Dot Playhouse, The Music Man with the New England Repertory Company, and Just People at the Long Wharf Theatre. He received a 1998–1999 Best Actor nomination from the Connecticut Critics' Awards for his character Stanley in Stamford Theater Work's A Rosen by Any Other Name. He had small roles in film and television shows, notably portraying Christopher Knight in the 2000 movie Growing Up Brady.

Ullman taught himself how to play the trombone. Ullman was selected to attend the Hugh O'Brian Youth Leadership Foundation (HOBY) youth leadership program of Connecticut in March 2002 where he was selected as the "outstanding ambassador" enabling him to attend the HOBY World Leadership Congress in the summer of 2002. He went to Hillel Academy and is a graduate of Fairfield Warde High School. He was active in National Conference of Synagogue Youth.

In 2004, Ullman starred as Sam in ABC Family's television film Searching for David's Heart. He appeared in a string of independent films including The Big Bad Swim and Driftwood (2006), Normal Adolescent Behavior (2007), and Prom Wars (2008). During this period, he guest-starred in TV shows like Big Love and Law & Order: Special Victims Unit.

While Ullman previously went by the stage name Ricky, he started using his real name in 2006 when he guest-starred on an episode of House and continued to use Raviv on all projects since.

After completing Rita Rocks in 2009, Ullman had some small roles in short films, television, and film but began to focus his career on theater acting, starring in Off-Broadway and regional theater productions.

From January 2012 to March 24, 2012, Ullman starred as the character Alex in the New Group's production of Russian Transport Off-Broadway in New York. On April 30, 2012, it was announced that Ullman joined the cast of Alena Smith's new Off-Broadway play The Bad Guys. It began performances on May 22 at the McGinn/Cazale Theater on the Upper West Side in New York City. In 2014, Ullman acted opposite of Holly Hunter and Richard Chamberlain in the first revival of Sticks and Bones. Sticks and Bones opened November 6, 2014 and closed December 14, 2014.

Ullman is an accomplished musician and plays drums and guitar. He has worked as a touring musician, notably with Soko on her 2012 tour. Ullman was a member of the band Reputante, which released their 2013 debut on Julian Casablancas' record label Cult Records. He departed the band sometime in 2015. With Reputante members James Levy and Jimmy Giannopoulos, Ullman helped form the band Lolawolf with Actress Zoë Kravitz. They released their self-titled debut EP on February 4, 2014. He left Lolawolf before the release of their first full-length record, Come Down, in October 2014.

Ullman's last professional acting credit was in 2015. However, he has acted since, starring in the 2017 benefit play Right Before I Go at The Town Hall.

In October 2016, Ullman announced his project Standing Ground, a documentary that explores the Standing Rock Sioux Nation, the North Dakota Access Pipeline and the fossil fuels industry. Production of the film took place throughout 2017.