While at SUNY Albany, Ostrovsky was a member of the rap trio Team Facelift, where he first started going by the name Fat Jew. He has said that the moniker was inspired by one of his counselors at Eisner Camp and by Screw magazine publisher Al Goldstein. Team Facelift formed in 2004, signed with Duck Down Records and released the album Mixed Emotions in 2006. They broke up in 2012.

In 2007, Ostrovsky was hired as an entertainment reporter interviewing celebrities on The Daily 10 on E! He later hosted the rap interview web series An Intimate Conversation. He sold a show parodying the Williamsburg, Brooklyn hipster culture to Amazon in 2013, which went unproduced. In 2014, he signed a deal to develop a scripted series for Comedy Central, based on his life. The deal was canceled in 2015. He signed with Creative Artists Agency in 2015, and that year began making regular appearances on Bloomberg News, starred in a Bud Light commercial that aired during Super Bowl XLIX, and appeared in the music video for DNCE's "Cake by the Ocean", as well as "Boys" by Charli XCX. He has appeared as himself in numerous reality television series, including 3AM and The Real Housewives of Beverly Hills. He has also played supporting roles in several films, including the upcoming teen comedy Status Update, starring Ross Lynch and Olivia Holt. He was featured in a 2017 ad campaign for Malibu Rum.

In 2009, Ostrovsky started an Instagram account with the handle "TheFatJewish". The account went viral in 2013 after Ostrovsky posted a video parodying Soul Cycle, in which he led a spin class for homeless people on Citi Bikes parked on the street. In 2013, his Instagram account was suspended due to inappropriate content. In response, he organized a rally held outside Instagram's New York office. The protest was live-streamed on Vice.com, and his account was restored after 15 minutes. In 2014 The New York Times wrote that "he has parlayed a profane sense of humor that mocks the tropes of social-media culture and the hipsters who propagate them; an apparent affection for pot, pets and grandparents; and his own slovenly, outlandish physical appearance… into a huge web following." Time magazine named Ostrovsky to its 2015 list of The 30 Most Influential People on the Internet. As of September 2017, he has over 10 million followers on Instagram.

Ostrovsky and Katie Sturino were married in 2014. They announced their divorce in 2017.

In 2015, Ostrovsky was caught posting screenshots of other comedians' jokes on Instagram and Twitter without their permission, and with the credits intentionally cropped out. After several stories were published about this behavior, Ostrovsky edited his past posts to credit the authors of the jokes.

In September 2016, Ostrovsky announced his Major Lazer tribute band Major Behavior, formed with DJ Maachew Bentley and billed as the world's first EDM tribute band. Major Behavior toured in 2016 as a part of Ostrovsky's IRL Tour. For the second IRL Tour in 2017, Major Behavior toured with Dillon Francis, RL Grime and Party Favor. Major Behavior was joined onstage by Major Lazer's Diplo during a May 2017 performance in Chicago.