Canteloube was born in Annonay, Ardèche, into a family with deep roots in the Auvergne region of France. He studied piano from the age of six with Amélie Doetzer, a friend of Frédéric Chopin. After earning his baccalauréat, he worked at a bank in Bordeaux. He returned to his family home in Malaret (Annonay) upon his father's death in 1896, remaining there until his mother's death in 1899 and then beyond as sole owner of the estate. After a period of silence and mourning, in 1901 Canteloube married Charlotte Marthe Calaret, who gave birth to twins Pierre and Guy in 1903.

He began studying with Vincent d'Indy via correspondence in 1901, reluctant to leave Malaret. Upon d'Indy's constant urging, he finally entered the Schola Cantorum in 1907 in Paris, where he remained until the beginning of World War I in 1914. At the Schola, Canteloube became close friends with fellow Composer and student Déodat de Séverac.

In 1907, he wrote a suite entitled Dans la montagne for piano and violin in four movements that was played at the Société Nationale. Other significant works followed, including Colloque sentimental for voice and string quartet (1908); Eglogue d'Automne for orchestra (1910); Vers la Princesse lointaine, a symphonic poem (1912); Aù printemps for voice and orchestra; and L'Arada (The Earth), a song cycle of six mélodies (1922).

Canteloube took more than thirty years (1924 to 1955) to complete the compilation of his most admired and famous collection of songs, Chants d'Auvergne. Passionate, sometimes to excess, the songs reflect the landscapes of the Auvergne in lush orchestral colors and have enabled French folklore and rustic melodies to become better known.

In 1925, Canteloube founded a group called La Bourrée with several young Auvergnats in Paris who were eager to publicize the folklore and the beauty of their home region. Canteloube himself believed that "peasant songs often rise to the level of purest art in terms of feeling and expression, if not in form" ("les chants paysans s’élèvent bien souvent au niveau de l'art le plus pur, par le sentiment et l'expression, sinon par la forme"). He composed several song collections, which include Chants de Haute-Auvergne, albums of songs of Rouergue, Limousin, and Quercy, regional religious songs (Chants religieux d'Auvergne), and L'Hymne des Gaules based on a poem by Philius Lebesque. He also participated in the creation of the Bardic College of Gaul.

Canteloube composed his first opera, Le mas ("The Farmstead" in Occitan language), to his own libretto from 1910 to 1925 (its composition delayed during war years). The three act work won the Prix Heugel in 1925, and was awarded the prize of 100,000 Francs. However, the reaction to this composition by the Leaders of the Opéra-Comique in Paris was far less enthusiastic than the jury. After pressure from the publisher, it finally premiered on 3 April 1929, but it was never revived. Vercingétorix, his second opera, in four acts, was inspired by a libretto by Étienne Clémentel, mayor of Riom (Puy-de-Dôme) and Hervé Louwyck on the Gauls' defeat by Julius Caesar. The Paris Opéra gave the first performance on 22 June 1933, but it was accused of lacking theatricality.

In 1941, he joined the government in Vichy France during the Nazi occupation, and wrote in the monarchist newspaper Action Française. He participated in numerous radio broadcasts of French folklore with his "Songs of France" with the tenor Christian Selva. The radio was an ideal vehicle for disseminating regional popular music.

Alongside his career as a Composer, Canteloube worked as a musicologist, collecting traditional French folksongs, which were published by Didier and Heugel. He also wrote biographies of Vincent d'Indy (1949) and of his friend Déodat de Séverac (1950).