The son of Vincenzo Leoncavallo, a police magistrate and judge, Leoncavallo was born in Naples on 23 April 1857. As a child, he moved with his father to the town of Montalto Uffugo in Calabria, where Leoncavallo lived during his adolescence. He later returned to Naples and was educated at the city's San Pietro a Majella Conservatory and later the University of Bologna studying literature under famed Italian poet Giosuè Carducci.

An agent located in the Rue du Faubourg-Saint-Denis secured Leoncavallo employment as an accompanist and instructor for artists who performed in Sunday concerts mostly at cafés. It was during this time that he met Berthe Rambaud (1863–1926) a "preferred student", who became his wife in 1895. Increasingly inspired by the French romantics, particularly Alfred de Musset, Leoncavallo began work on a symphonic poem based on Musset's poetry entitled "La nuit de mai". The work was completed in Paris in 1886 and premiered in April 1887 to critical acclaim. With this success and now with enough accumulated money Leoncavallo and Rambaud would return to Milan to begin his career as a Composer of opera.

In 1879 Leoncavallo's uncle Giuseppe, Director of the press department at the Foreign Ministry in Egypt, suggested that his young nephew come to Cairo to showcase his pianistic abilities. Arriving shortly after the deposition of Khedive Ismail, Leoncavallo eventually secured work as a piano Teacher and Pianist to the brother of the new Khedive Tewfik Pasha. His time in Egypt concluded abruptly in 1882 after revolts in Alexandria and Cairo led by ‘Urabi in which the Composer quickly departed for France. In Paris, Leoncavallo found lodging in Montmartre.

Back in Italy, Leoncavallo spent some years teaching and attempting ineffectively to obtain the production of more than one opera, notably Chatterton. In 1890 he saw the enormous success of Pietro Mascagni's Cavalleria rusticana and wasted no time in producing his own verismo work, Pagliacci. (According to Leoncavallo, the plot of this work had a real-life origin: he claimed it derived from a murder trial, in Montalto Uffugo, over which his father had presided.)

Pagliacci was performed in Milan in 1892 with immediate success; today it is the only work by Leoncavallo in the standard operatic repertory. Its most famous aria "Vesti la giubba" ("Put on the costume" or, in the better-known older translation, "On with the motley") was recorded by Enrico Caruso and laid claim to being the world's first record to sell a million copies (although this is probably a total of Caruso's various versions of it made in 1902, 1904 and 1907).

The next year his I Medici was also produced in Milan, but neither it nor Chatterton (belatedly produced in 1896)—both early works—obtained much lasting favour. Much of Chatterton, however, was recorded by the Gramophone Company (later HMV) as early as 1908, and remastered on CD almost 100 years later by Marston Records. Leoncavallo himself conducts the performance or at very least supervises the production.

It was not until Leoncavallo's La bohème was performed in 1897 in Venice that his talent obtained public confirmation. However, it was outshone by Puccini's opera of the same name and on the same subject, which was premiered in 1896. Two tenor arias from Leoncavallo's version are still occasionally performed, especially in Italy.

Subsequent operas by Leoncavallo were in the 1900s: Zazà (the opera of Geraldine Farrar's famous 1922 farewell performance at the Metropolitan Opera), and 1904's Der Roland von Berlin. In 1906 the Composer brought Singers and orchestral Musicians from La Scala to perform concerts of his music in New York, as well as an extensive tour of the United States. The tour was, all in all, a qualified success. He had a brief success with Zingari which premiered in Italian in London in 1912, with a long run at the Hippodrome Theatre. Zingari also reached the United States but soon disappeared from the repertoire.

Leoncavallo also composed songs, most famously "Mattinata", which he wrote for the Gramophone Company (which became HMV) with Caruso's unique voice in mind. On 8 April 1904, Leoncavallo accompanied Caruso at the piano as they recorded the song. On 8 December 1905 he recorded five of his own pieces for the reproducing piano Welte-Mignon.

Leoncavallo died in Montecatini Terme, Tuscany, on 9 August 1919. His funeral was held two days later, with hundreds in attendance, including fellow Composer Pietro Mascagni and longtime rival Giacomo Puccini. He was buried in the Cimitero delle Porte Sante in Florence.

From the 1970s Edipo re has had a number of revivals, both as concert performances (including Rome 1972, Concertgebouw (Amsterdam) 1977 and Konzerthaus, Vienna 1998) as well as fully staged productions at the Teatro Regio, Turin, in 2002 and the Thessaloniki Opera 2008. It remains to be seen who will be given the credit for this opera in Future revivals.

The Museo Leoncavallo (Leoncavallo Museum) was established in 2002 in Brissago to commemorate the Composer with personal items and original manuscripts on display including statues representing characters from his other operas including Zazà and Der Roland von Berlin.

Today he remains largely known for Pagliacci, one of the most popular works in the repertory, appearing as number 20 on the Operabase list of the most-performed operas worldwide in the 2013/14 season. His other well-known works include the song "Mattinata", popularized by Enrico Caruso, as well as the symphonic poem La nuit de mai.

After a series of operettas, Leoncavallo appeared to have tried for one last serious effort with Edipo re (it). It had always been assumed that Leoncavallo had finished the work but had died before he could finish the orchestration, which was completed by Giovanni Pennacchio (it). However, with the publication of Konrad Dryden's biography of Leoncavallo it was revealed that Leoncavallo may not have written the work at all (although it certainly contains themes by Leoncavallo). A review of Dryden's study notes: "That fine Edipo re ... was not even composed by [Leoncavallo]. His widow paid another Composer to concoct a new opera using the music of Der Roland von Berlin. Dryden didn't find one reference to the opera in Leoncavallo’s correspondence nor is there a single note by him to be found in the handwritten score."