While little is known about his early life, a Flemish origin along with a French upbringing has been suggested from variations on the spelling of his name, and he may originally have been from the vicinity of Liège or Namur, in present-day Belgium. He moved to Italy as a young man, and was present in Florence by the late 1520s, therefore having an opportunity to meet or work with Philippe Verdelot, who wrote the earliest named madrigals. In or immediately before 1538 he moved to Rome where he obtained an appointment with the papal choir at St. Peter's Basilica; many composers from the Netherlands served as Singers there throughout this era, and it is even possible that he went to Rome before coming to Florence. Still in Rome, in January 1539, he probably was made a member of the Julian Chapel (the records give his name as "Jacobus flandrus", suggesting a Flemish origin, but it cannot be known with certainty if this record referred to Arcadelt). After some months there he became a member of the Sistine Chapel, where he was appointed "magister puerorum." The same year saw the publication of no less than four books of his madrigals. The first of these collections, Il primo libro di madrigali, went through 45 editions, becoming the most widely reprinted collection of madrigals of the time.

Antoine Gardano became the primary Italian publisher for Arcadelt, although the competing Venetian publishing house of Scotto brought out one of his madrigal books as well. Arcadelt's Il bianco e dolce cigno opened one of Gardano's books; as the piece had already achieved immense fame, it was the main selling point. In Paris, some of Arcadelt's chansons appeared as early as 1540 in the publications of Pierre Attaingnant and Jacques Moderne, so must have been written in Italy. After Arcadelt returned to France, his chansons, masses, and motets appeared in the editions of the printing firm of Le Roy and Ballard throughout the 1550s and 1560s, while his music was still being printed in distant Venice.

Arcadelt wrote over 200 madrigals before he left Italy in 1551 to return to France, where he spent the remainder of his life; his numerous chansons date from this and subsequent years. In 1557 he published a book of masses, dedicated to his new employer, Charles de Guise, Cardinal of Lorraine (Arcadelt was maître de chapelle, i.e. choirmaster for him). In this publication he was mentioned as a member of the royal chapel, and therefore must have served both Henry II (died 1559) and Charles IX during this late phase of his career. In Paris he employed the publishing house of Le Roy and Ballard, who printed his abundant chansons, masses and motets just as the Venetian printers had earlier printed his madrigals.

His music became immensely popular in Italy and France for more than a hundred years, with his first book of madrigals being reprinted fifty-eight times by 1654, and his music appearing in innumerable intabulations for instruments such as the lute, guitar, and viol. Additional hints to his popularity are the frequency with which anonymous compositions were attributed to him, and the appearance of his music in several paintings of Musicians from the time. Likely his popularity was due to his gift for capturing the Italian spirit and marrying it with the technical perfection of the Franco-Flemish harmonic and polyphonic style; in addition he wrote catchy tunes which were easy to sing. Unlike later generations of madrigal composers, Arcadelt did not expect professional Singers to be the only consumers of his work; anyone who could read notes could sing his madrigals.

A complete modern edition of Arcadelt's works is published in CMM, xxxi, 1–10 (ten volumes), edited by Albert Seay. The first volume contains Arcadelt's masses; his secular compositions are in volumes two through nine, and his motets and other sacred music are in volume ten. Below is an partial list of his works. Note that numbering is by number of voices: for Example, there is an Il primo libro di madrigali (First Book of Madrigals) for four voices, and another Primo libro di madrigali for three.

For his texts, Arcadelt chose poets ranging from Petrarch (and his setting of a complete canzone, as a set of five interrelated madrigals, was the predecessor of the vogue for madrigal cycles), Pietro Bembo, Sannazaro, to Florentines Lorenzino de'Medici, Benedetto Varchi, Filippo Strozzi, and Michelangelo himself, to others such as Luigi Cassola of Piacenza, a now-obscure Writer who was among the most often-set poets of the early madrigalists. Much of the poetry of Arcadelt's madrigals has remained anonymous, just as some of Arcadelt's music is believed to survive anonymously. Another poet he set was the Marquis Alfonso d'Avalos, who wrote the words to his most single famous composition, and one of the most enduring of the entire 16th century: the four-voice madrigal Il bianco e dolce cigno (The white and gentle swan).