Shemp was born as Samuel Horwitz in Manhattan, New York on March 11, 1895, and raised in Brooklyn. He was the third-born of the five Horwitz brothers, the sons of their Lithuanian Jewish parents: Solomon Horwitz (1872–1943) and Jennie Horwitz (1870–1939). Irving and Jack were his older brothers; Moses and Jerome were his younger brothers.

By 1922, Moe had teamed up with boyhood-friend-turned-vaudeville star Ted Healy in a "roughhouse" act. One day Moe spotted his brother Shemp in the audience, and yelled at him from the stage. Quick-witted Shemp yelled right back, and walked onto the stage. From then on he was part of the act, usually known as "Ted Healy and His Stooges". His original stooges were the Howard brothers, with Larry Fine joining in March 1925. On stage, Healy sang and told jokes while his three noisy stooges got in his way. Healy retaliated with physical and Verbal abuse. Shemp played a bumbling fireman in the Stooges' first film, Soup to Nuts (1930), the only film in which he plays one of Healy's gang.

In September 1925, Shemp (at age 30) married Gertrude Frank (26), a fellow New Yorker. They had one child, Morton (1926–1972).

After a disagreement with Healy in August 1930, Moe, Larry and Shemp left to launch their own act, "Howard, Fine & Howard," and joined the RKO vaudeville circuit. The three premiered at Los Angeles' Paramount Theatre on August 28, 1930. In 1931, they added "Three Lost Soles" to the act's name, and took on Jack Walsh as their straight man. Moe, Larry and Shemp continued until July 1932, when Ted Healy approached them to team up again for the Shuberts' Broadway revue "Passing Show of 1932," and they readily accepted the offer. In spite of any differences, Moe knew that an association with the nationally-known Ted Healy provided opportunities the three comics were not achieving on their own.

On August 16, 1932, in a contract dispute, Healy walked out on the Shuberts' revue during rehearsals. Following this, three days later, tired of what he considered Healy's domineering handling of the Stooges' career, Shemp left Healy's act to remain with "Passing Show," which closed in September during roadshow performances and after pan reviews in Detroit and Cincinnati. Shemp regrouped to form his own act and played on the road for a few months. He landed at Brooklyn's Vitaphone Studios for movie appearance opportunities in May 1933. When he split from Healy, Shemp was immediately replaced by his and Moe's younger brother Jerry Howard (known as Curly) in the act.

Shemp Howard, like many New York-based performers, found work at the Vitaphone studio in Brooklyn. Originally playing bit roles in Vitaphone's Roscoe Arbuckle comedies, showing off his goofy appearance, he was entrusted with speaking roles and supporting parts almost immediately. He was featured with Vitaphone comics Jack Haley, Ben Blue and Gus Shy, then co-starred with Harry Gribbon, Daphne Pollard and Johnnie Berkes, and finally starred in his own two-reel comedies. A Gribbon-Howard short, Art Trouble (1934), also features then unknown James Stewart in his first film role.

Shemp would seldom stick to the script, and would liven up a scene with ad-libbed incidental dialogue or wisecracks. This became a trademark of his performances. In late 1935, Vitaphone was licensed to produce short comedies based on the "Joe Palooka" comic strip. Shemp was cast as "Knobby Walsh," and though only a supporting character became the comic focus of the series, with Johnny Berkes and Lee Weber as his foils. He costarred in the first seven shorts, released during 1936 and 1937; nine were produced altogether, with the last two done after Shemp's departure from Vitaphone for greener pastures on the West Coast.

Away from Vitaphone he attempted, unsuccessfully, to lead his own group of "stooges" in the Van Beuren musical comedy short The Knife of the Party. But Shemp's solo career was otherwise successful. He followed his brothers' lead, moved to the West Coast in 1937 and landed supporting-actor roles at several studios, predominantly at Columbia Pictures and Universal Studios. Shemp worked exclusively at Universal from August 1940 to August 1943. He performed with such comics as W. C. Fields, with whom he appeared in the film The Bank Dick (1940) playing the bartender; and with duo-comedians Abbott and Costello and Olsen and Johnson. He also lent comic relief to Charlie Chan and The Thin Man murder mysteries, and was in several Universal B-musicals of the early 1940s, among them Strictly in the Groove (1942), How's About It? (1943), Moonlight and Cactus (1944) and San Antonio Rose (1941), in which he is paired with Lon Chaney, Jr. as a faux Abbott and Costello. Most of these took great advantage of his improvisational skills. He was briefly teamed with comedians Billy Gilbert and Maxie Rosenbloom for three B-comedy features in 1944–45. He also played a few serious parts, such as his supporting role in Pittsburgh (1942), starring Marlene Dietrich and John Wayne.

During 1938–1939 and 1944–1946, Shemp appeared in Columbia's two-reel comedies, co-starring with Columbia regulars Andy Clyde, The Glove Slingers, El Brendel and Tom Kennedy. He was given his own starring series in 1944; he was working for Columbia in this capacity when his brother Curly was felled by a debilitating stroke on May 6, 1946, although he had already suffered a series of them prior to the filming of If a Body Meets a Body (1945). Shemp replaced Curly in Columbia's popular Stooge shorts, knowing that Moe and Larry would be out of work if he refused. He rejoined the Stooges at first on only a temporary basis until Curly recovered, but as Curly's condition worsened it became apparent that Shemp's association with the Stooges would be permanent. Although speculated that rejoining the Stooges was initially done reluctantly, Shemp's income increased significantly over the next several years due to this career change. Before replacing Curly in the film series, it has also been reported that Shemp substituted for his brother in some personal appearances in the early 1940s, but that actually only occurred once in January 1945 for a week-long appearance at the St. Charles Theatre in New Orleans. Curly died on January 18, 1952, at the age of 48.

On November 22, 1955, Shemp went out with his friends Al Winston and Bobby Silverman to a boxing match (one of Shemp's favorite pastimes), at the old Hollywood Legion Stadium (at North El Centro Avenue & Selma Avenue), located just one block above the Hollywood Palladium. While returning home in a taxicab that evening, he died of a sudden massive heart attack, at the age of 60. He was leaning back and lighting a cigar after telling a joke, when he suddenly slumped over on his friend Al Winston's lap, accidentally burning Al with the cigar. Al thought Shemp was playing a joke, since Shemp was laughing moments earlier, but realized he was actually dead. Moe's autobiography gives a death date of November 23, 1955, as do most subsequent accounts, because of Moe's book. But much of that book was finished posthumously by his daughter and son-in-law, and some specific details were confused as a result. The Los Angeles county coroner's death certificate states that Shemp Howard died on Tuesday, November 22, 1955, at 11:35 [PM] PST; confirming that, Shemp's obituary appeared in the November 23 afternoon editions of Los Angeles newspapers, establishing the night of November 22 as the date of death. A different account is offered by former daughter-in-law Geri Greenbaum, who says Shemp's death happened just as their cab came over the rise on Barham Boulevard (heading to his Toluca Lake home).

The re-edited films range from clever to blatantly patchy, and are often dismissed as second-rate. Rumpus in the Harem borrows from Malice in the Palace, Hot Stuff from Fuelin' Around, and Commotion on the Ocean (all re-edits released 1956) from Dunked in the Deep (originals all 1949). The best-received (and most technically accomplished) is Scheming Schemers (again 1956), combining new footage with recycled clips from three old Stooge shorts: A Plumbing We Will Go (1940), Half-Wits Holiday (1947) and Vagabond Loafers (1949).

When it was time to renew the Stooges' contract, Columbia hired Comedian Joe Besser to replace Shemp. Columbia stopped filming new Stooge shorts for theaters in December 1957, but kept the series going into the 1960s by reissuing Shemp's Stooge shorts, so that Shemp Howard remained a popular star for more than a decade after he died.

The Three Stooges earned a star on the Hollywood Walk of Fame on August 30, 1983.

In a TV biopic film, The Three Stooges (2000), produced by Mel Gibson, Shemp was portrayed by 'fake Shemp' Johnny Kassir, who donned a floppy, straight-haired wig to portray the famous comic.

Shemp appeared with Moe and Larry in 73 short subjects—or 77, counting four that were made after Shemp's death using stock footage—and in the feature film Gold Raiders (1951). He suffered a mild stroke in November 1952, although he recovered from it within weeks and without noticeable effect on his remaining films with the Stooges which were by then often remakes of earlier films that used recycled footage to reduce costs.