Born Philip Silver or Philip Silversmith on May 11, 1911, in Brooklyn, New York, in the working-class Brownsville section, he was the eighth and youngest child of Russian Jewish immigrants, Saul and Sarah (née Handler) Silver. His siblings were Lillian, Harry, Jack, Saul, Pearl, Michael, and Reuben Silver. His father, a sheet metal worker, helped build the early New York skyscrapers.

Silvers next worked in short films for the Vitaphone studio, such as Ups and Downs (1937), and on Broadway, where he made his début in the short-lived show Yokel Boy in 1939. Critics raved about Silvers, who was hailed as the bright spot in the mediocre play. The Broadway revue High Kickers (1941) was based on his concept.

He made his feature film début in Hit Parade of 1941 in 1940 (his previous appearance as a 'pitch man' in Strike Up the Band was cut). Over the next two decades, he worked as a character actor for Metro-Goldwyn-Mayer, Columbia, and 20th Century Fox, in such films as All Through the Night (1942) with Humphrey Bogart. That same year, he played a scene with W. C. Fields in Tales of Manhattan which was cut from the original release, but restored decades later in home video issues. Silvers also appeared in Lady Be Good (1941), Coney Island (1943), Cover Girl (1944) and Summer Stock (1950). When the studio system began to decline, he returned to the stage. Cover Girl demonstrates that Silvers was an accomplished Dancer, performing flawlessly in quick-tempo trios with famed Dancers Gene Kelly and Rita Hayworth.

Silvers wrote the lyrics for Frank Sinatra's "Nancy (With the Laughing Face)". Although he was not a Songwriter, he wrote the lyrics while visiting Composer Jimmy Van Heusen. The two composed the song for Van Heusen's writing partner Johnny Burke, for his wife Bessie's birthday. Substituting Sinatra's little daughter's name Nancy at her birthday party, the trio pressed the singer to record it himself. The song became a popular hit in 1945 and was a staple in Sinatra's live performances. Towards the end of the Second World War, Silvers entertained the troops during several successful overseas USO tours with Sinatra.

Silvers scored a major triumph in Top Banana, a Broadway show of 1952. Silvers played Jerry Biffle, the egocentric, always-busy star of a major television show. (The character is said to have been based on Milton Berle.) Silvers dominated the show and won a Tony Award for his performance. He repeated the role in the 1954 film version that was originally released in 3-D.

Silvers became a household name in 1955 when he starred as Sergeant Ernest G. Bilko in You'll Never Get Rich, later retitled The Phil Silvers Show. The military comedy became a television hit, with the opportunistic Bilko fast-talking his way through one obstacle after another. In 1958, CBS switched the show to be telecast on Friday nights and moved the setting to Camp Fremont in California. A year later, the show was off the schedule.

Silvers returned to Broadway in the musical Do Re Mi in December 1960, receiving a nomination for the Tony Award for Best Performance by a Leading Actor in a Musical. Stanley Green wrote, "It was particularly blessed by offering two outstanding clowns in Phil Silvers as the pushiest of patsies and Nancy Walker." Throughout the 1960s, he appeared in films such as It's a Mad, Mad, Mad, Mad World (1963) and 40 Pounds of Trouble (1963). He was featured in Marilyn Monroe's last film, the unfinished Something's Got to Give (1962). In the 1963–1964 television season, he appeared as Harry Grafton, a factory foreman interested in get-rich-quick schemes, much like the previous Bilko character, in CBS's 30-episode The New Phil Silvers Show, with co-stars Stafford Repp, Herbie Faye, Buddy Lester, Elena Verdugo as his sister, Audrey, and her children, played by Ronnie Dapo and Sandy Descher. In 1967, he starred as a guest in one of the British Carry On films, Follow That Camel, a Foreign Legion parody in which he played a variation of the Sergeant Bilko character, Sergeant Nocker. Producer Peter Rogers employed him to ensure the Carry On films' success in America, though Silvers' presence did not ensure the film's success on either side of the Atlantic. His salary was £30,000, the largest Carry On salary ever, only later met by the appearance of Elke Sommer in Carry On Behind.

Like his alter-ego Ernie Bilko, Silvers was a compulsive gambler, and also suffered from chronic depression. He suffered a nervous breakdown in 1962 while performing in Spain. While staying in Reno, Nevada, in the 1950s, he would often gamble all night. On one occasion, at the tiny Cal-Neva Lodge in nearby Lake Tahoe, Nevada, Silvers spent an entire night playing craps until he lost all his money and then went through $1,000 in credit. A taxi was called to return him to Reno. It was "[o]ne of the worst nights of my life', Silvers told the driver, adding, "Don't wait for any Lights and don't wait for any tip . . . I left it at the Cal-Neva!"

Silvers also guested on The Beverly Hillbillies, and various TV variety shows such as The Carol Burnett Show, Rowan & Martin's Laugh-In and The Dean Martin Show. He appeared as curmudgeonly Hollywood Producer Harold Hecuba in the classic 1966 episode "The Producer" on Gilligan's Island, where he and the castaways performed a musical version of Hamlet. (Silvers' production company Gladasya – named after his catchphrase "Gladdaseeya!" – financed the show.)

Silvers suffered a stroke during the run of A Funny Thing Happened on the Way to the Forum in August 1972. He was left with slurred speech. Despite his poor health, he continued working, playing Harry Starman in the 1974 "Horror in the Heights" episode of Kolchak: The Night Stalker starring Darren McGavin. His guest appearances continued into the early 1980s, including co-starring in The Chicken Chronicles (1977), an appearance on Fantasy Island as an old comic trying to reunite with his old partner, and on Happy Days as the father of Jenny Piccolo (played by his daughter Cathy Silvers). In 1978, Silvers played the cab driver "Hoppy" in Neil Simon's send-up of hard-boiled detective films, The Cheap Detective, which starred Peter Falk. In his cab, Silvers can be heard (three words) and seen turning his head towards the camera and breaking into a smile (1/4 fps) at the movie's ending immediately prior to Falk entering "Hoppy's" cab. His final appearance was in an episode of CHiPs (entitled "Hot Date") in 1983.

On November 1, 1985, Silvers died in his sleep in Century City, California. According to his family, Silvers died of natural causes. He was interred at Mount Sinai Memorial Park Cemetery in Los Angeles.

Voice actor Daws Butler employed an impression of Silvers as the voice of the Hanna-Barbera cartoon character Hokey Wolf and also used the same voice in numerous cartoons for Jay Ward. The premise of The Phil Silvers Show was the basis for the Hanna-Barbera cartoon Top Cat, for which Arnold Stang moderately imitated Silvers' voice for the title character. The 1993 cartoon series The Adventures of Sonic the Hedgehog featured a character called Wes Weasley, who had a very similar appearance and voice to Silvers.

In 1996, TV Guide ranked him number 31 on its 50 Greatest TV Stars of All Time list

In 2003, The Phil Silvers Show was voted Best Sitcom in the Radio Times Guide to TV Comedy. In a 2005 poll to find The Comedian's Comedian, Silvers was voted #42 on the list of the top 50 comedy acts ever by fellow comedians and comedy insiders. Dick Van Dyke, who made his TV debut on Bilko, says he "was always fascinated with Phil's sense of timing. Incredible."

Phil Silvers was married twice, to Jo-Carroll Dennison, and to Evelyn Patrick. Both of his marriages ended in divorce. He had five daughters — Candace, Cathy, Laurie, Nancey, and Tracey Edythe — all by his second wife, Evelyn Patrick.