In 1910 at the age of 19, he began a short career playing baritone in and directing circus bands. That year, he joined the Robinson Famous Shows under Conductor Woodring Van Anda ("Woody Van"). The next year he was performing in the Yankee Robinson Circus band under Theo. Stout. In 1912, he performed in the Sells-Floto Circus under W.P. English (a famous march composer), and in 1913 in the Barnum and Bailey band under Ned Brill. At the request of Brill he wrote (and dedicated to Brill) "Barnum & Bailey's Favorite", his most famous march and possibly the most recognizable American music written specifically for the circus. It would soon be adopted as the theme of the circus.

"Barnum and Bailey's Favorite" (1913) remains his best-known composition, but other pieces that retain their popularity among fans of band music include:

His first full-time conducting job was in 1914 through 1915 with the Sells Floto Circus and Buffalo Bill's Wild West Show combined shows band. He became bandmaster for the Sells-Floto Circus in 1915 and was bandmaster of the Barnum and Bailey Circus band 1917–1918. In his final band, he included his wife Ruth (Lovett) as the calliope performer. He had married Ruth November 17, 1916.

After a year in Canton where he directed the Grand Army Band (1919) King settled down in Fort Dodge, Iowa. This was in 1920 (age 29) and for the next fifty-one years he conducted the Fort Dodge Municipal Band, which featured Future American Bandmasters Association President Joseph Hermann on clarinet. The band became known as King's Band.

King was instrumental in the passage of the Iowa Band Law in 1921, which allowed cities to levy a local tax for maintenance of a band. He commemorated this with one of his marches, "Iowa Band Law". In 1960, King would direct "Iowa Band Law" with the largest mass band ever assembled: 188 high school bands and nearly 13,000 Musicians at a nationally televised University of Michigan football game.

A physical description of Karl King in the 1951 Who's Who in Music: brown eyes, brown hair, 6’1” in height, 200 pounds.

King's final published march was "The Home Town Boy March" (1962) dedicated to Meredith Willson.

A disastrous fire on January 12, 1971 destroyed the Knights of Columbus Hall where the Fort Dodge Band held rehearsals and stored instruments and uniforms. The hall also held most of Karl King's original manuscripts, including special arrangements.

King's marches for circus bands are usually composed at a high difficulty level (grade 4–5 typically) American march music. He also contributed greatly to the school band movement with numerous compositions at various levels of difficulty.