Brown was born on July 28, 1891, in Holgate, Ohio, near Toledo, into a large family largely of Welsh descent. He spent most of his childhood in Toledo. In 1902, at the age of ten, he joined a troupe of circus tumblers known as the Five Marvelous Ashtons, who toured the country on both the circus and vaudeville circuits. Later he became a professional baseball player. Despite his skill, he declined an opportunity to sign with the New York Yankees to pursue his career as an entertainer. After three seasons he returned to the circus, then went into Vaudeville and finally starred on Broadway. He gradually added comedy to his act, and transformed himself into a Comedian. He moved to Broadway in the 1920s, first appearing in the musical comedy Jim Jam Jems.

Brown married Kathryn Francis McGraw in 1915. The marriage would last until his death in 1973. The couple had four children: two sons, Don Evan Brown (December 25, 1916 – October 8, 1942; Captain in the United States Army Air Force, who was killed in the crash of an A-20B Havoc bomber while serving as a ferry pilot) and Joe LeRoy "Joe L." Brown (September 1, 1918 – August 15, 2010), and two daughters, Mary Katherine Ann (b. 1930) and Kathryn Francis (b. 1934). Both daughters were adopted as infants.

In late 1928, Brown began making films, starting the next year with Warner Bros.. He quickly became a favorite with child audiences, and shot to stardom after appearing in the first all-color all-talking musical comedy On with the Show (1929). He starred in a number of lavish Technicolor Warner Brothers musical comedies including: Sally (1929), Hold Everything (1930), Song of the West (1930), and Going Wild (1930). By 1931, Joe E. Brown had become such a star that his name was billed above the title in the films in which he appeared.

Brown was a Sports enthusiast, both in film and personally. Some of his best films were the "baseball trilogy" which consisted of Fireman, Save My Child (1932), Elmer the Great (1933) and Alibi Ike (1935). He was also a television and radio broadcaster for the New York Yankees in 1953. His son, Joe L. Brown, inherited an interest in baseball, becoming the general manager of the Pittsburgh Pirates for more than twenty years. Brown also spent Ty Cobb's last days with him before he died, discussing his life.

He was caricatured in the Disney cartoons Mickey's Gala Premiere (1933), Mother Goose Goes Hollywood (1938), and The Autograph Hound (1939); all of them contain a scene in which he is seen laughing so loud that his mouth opens extremely wide. According to the official autobiography Daws Butler: Characters Actor, Daws Butler used Joe E. Brown as inspiration for the voices of two Hanna-Barbera cartoon characters: Lippy the Lion (1962) and Peter Potamus (1963–1966).

Brown was one of the few vaudeville comedians to appear in a Shakespeare film; he played Francis Flute in the Max Reinhardt/William Dieterle film version of Shakespeare's A Midsummer Night's Dream (1935) and was highly praised for his performance. He starred in Polo Joe (1936) with Carol Hughes and Richard "Skeets" Gallagher, and in Sons o' Guns. In 1933 and 1936, he became one of the top ten earners in films. He was sufficiently well known internationally by this point to be depicted in comic strips in the British comic Film Fun for twenty years from 1933.

He left Warner Brothers to work for Producer David L. Loew, starring in When's Your Birthday? (1937). In 1938, he starred in The Gladiator, a loose film adaptation of Philip Gordon Wylie's 1930 novel Gladiator that influenced the creation of Superman. He gradually switched to making "B" pictures.

In 1939, Brown testified before the House Immigration Committee in support of a bill that would allow 20,000 German Jewish refugee children into the US. He later adopted two refugee children.

Joe E. Brown's other two sons were in the military Service. In 1942 Brown's son, Captain Don E. Brown, was killed when his A-20 Havoc crashed near Palm Springs, California. At 50, Brown himself was too old to enlist, but he traveled thousands of miles at his own expense to entertain American troops. He was the first to do so, traveling to both the Caribbean and Alaska before Bob Hope had, and before the USO was organized.

In 1948, he was awarded a Special Tony Award for his work in the touring company of Harvey.

Another of his notable postwar roles was that of "Cap'n Andy Hawkes" in MGM's 1951 remake of Show Boat, a role that he reprised onstage in the 1961 New York City Center revival of the musical and on tour. The musical film version included such prominent costars as Ava Gardner, Howard Keel, and Kathryn Grayson. Brown performed several dance routines in the film, and famed Choreographer Gower Champion appeared along with first wife Marge. Brown's final film appearance was in The Comedy of Terrors (1964). Weeks earlier, he had appeared as Diamond "Dimey" Vine in an episode of Jack Palance's ABC circus drama The Greatest Show on Earth.

Brown's Sports enthusiasm also led to him becoming the first President of PONY Baseball and Softball (at the time named Pony League) when the organization was incorporated in 1953. He continued in the post until late 1964, when he retired. Later he traveled additional thousands of miles telling the story of PONY League, hoping to interest adults in organizing baseball programs for young people. He was also a fan of Thoroughbred horse racing, a regular at Del Mar Racetrack and the races at Santa Anita.

Joe L. Brown shared his father's love of baseball, serving as general manager of the Pittsburgh Pirates from 1955 to 1976, and briefly in 1985, also building the 1960 and 1971 World Series champions. Brown's '71 Pirates featured baseball's first all-black starting nine.

His best known postwar role was that of aging millionaire Osgood Fielding III in Billy Wilder's 1959 comedy, Some Like It Hot. Fielding falls for Daphne (Jerry), played by Jack Lemmon in drag; at the end of the film, Lemmon takes off his wig and reveals to Brown that he is a man, to which Brown responds; "well, nobody's perfect", one of the most celebrated punchlines in film history.

For his contributions to the film industry, Brown was inducted into the Hollywood Walk of Fame in 1960 with a motion pictures star located at 1680 Vine Street.

In 1961, Bowling Green State University renamed the theatre in which Brown appeared in Harvey in the 1950s as the "Joe E. Brown Theatre". It was closed in 2011.

Brown began having heart problems in 1968, after suffering a severe heart attack, and underwent cardiac surgery. He died from arteriosclerosis on July 6, 1973, at his home in Brentwood, Los Angeles, California, three weeks before his 82nd birthday. He is interred at Forest Lawn Memorial Park in Glendale, California.

Rose Naftalin's popular 1975 cookbook includes a cookie named the "Joe E. Brown". Brown was a frequent customer of Naftalin's Toledo restaurant.

"While big USO names like Bob Hope did not visit the Leyte, Philippine area my father was in (housing was not good), Leyte received any number of entertainers during the war. Dad said the entertainers were all just a bunch of nice people. One group in particular was the top actors from a very popular and large musical of the times. Dad and a few of his buddies would walk back to the tents that housed the USO performers and would visit with them. Mostly, the USO performers were curious about the events on the islands and how the men were handling things. One performer, a "wonderful comedian" named Joe E. Brown, would commandeer a military vehicle and be driven around the island. The entertainer would stop military pedestrians, "ream" them for some inconsequential matter, start laughing, then invite them into the cab so they could be driven to their destinations. Joe E. Brown was greatly appreciated." (Memories of Capt. Donald Courtright, told to his daughter Mary in 2011.)