Born in Baltimore, Maryland, the son of Alexander and Mary Muse, he studied at Dickinson College, Carlisle, Pennsylvania, and received an international law degree in 1911. He was acting in New York by the 1920s, during the Harlem Renaissance with two Harlem theatres, Lincoln Players and Lafayette Players.

Muse moved to Chicago for a while, and then moved to Hollywood and performed in Hearts in Dixie (1929), the first all-black movie. For the next fifty years, he worked regularly in minor and major roles. While with the Lafayette Players, Muse worked under the management of Producer Robert Levy on productions that helped black actors to gain prominence and respect. In regards to the Lafayette Theatre's staging of Dr. Jekyll and Mr. Hyde, Muse said the play was relevant to black actors and audiences "because, in a way, it was every black man's story. Black men too have been split creatures inhabiting one body.". Muse appeared as an opera singer, minstrel show performer, vaudeville and Broadway actor; he also wrote songs, plays, and sketches. In 1943, he became the first African American Broadway Director with Run Little Chillun.

Muse was also the co-writer of several notable songs. In 1931, with Leon René and Otis René, Muse wrote "When It's Sleepy Time Down South", also known as "Sleepy Time Down South". The song was sung by Nina Mae McKinney in the movie Safe in Hell (1931), and later became a signature song of Louis Armstrong.

He was the major star in Broken Earth (1936), which related the story of a black sharecropper whose son miraculously recovers from fever through the father's fervent prayer. Shot on a farm in the South with nonprofessional actors (except for Muse), the film's early scenes focused in a highly realistic manner on the incredible hardship of black farmers, with plowing scenes. In 1938, Muse co-starred with boxer Joe Louis in Spirit of Youth, the fictional story of a champion boxer which featured an all black cast. Muse and Langston Hughes wrote the script for Way Down South (1939).

Muse performed in Broken Strings (1940), as a concert Violinist who opposes the Desire of his son to play "swing". From 1955-56, Muse was a regular on the weekly TV version of Casablanca, playing Sam the Pianist (a part he was under consideration for in the original Warner Brothers film), and in 1959, he played Peter, the Honey Man, in Porgy and Bess.

He appeared on Disney's TV miniseries The Swamp Fox. Other film credits include Buck and the Preacher (1972), The World's Greatest Athlete (1973) and as Gazenga's Assistant, "Snapper" in Car Wash (1976). His last acting role was in The Black Stallion (1979).

He received an honorary Doctor of humanities degree from Bishop College, Dallas, Texas, in 1972, and was a member of Phi Beta Sigma fraternity, Omega Chapter. Muse died in Perris, California, on October 13, 1979, one day before his 90th birthday and the same day that his final film was released.