Cohen-Tannoudji was born in Constantine, French Algeria, to Algerian Jewish parents Abraham Cohen-Tannoudji and Sarah Sebbah. When describing his origins Cohen-Tannoudji said: "My family, originally from Tangier, settled in Tunisia and then in Algeria in the 16th century after having fled Spain during the Inquisition. In fact, our name, Cohen-Tannoudji, means simply the Cohen family from Tangiers. The Algerian Jews obtained the French citizenship in 1870 after Algeria became a French colony in 1830."

After finishing secondary school in Algiers in 1953, Cohen-Tannoudji left for Paris to attend the École Normale Supérieure. His professors included Henri Cartan, Laurent Schwartz, and Alfred Kastler.

In 1958 he married Jacqueline Veyrat, a high school Teacher, with whom he has three children. His studies were interrupted when he was conscripted into the army, in which he served for 28 months (longer than usual because of the Algerian War). In 1960 he resumed working toward his doctorate, which he obtained from the École Normale Supérieure under the supervision of Alfred Kastler and Jean Brossel at the end of 1962.

After his dissertation, he started teaching quantum mechanics at the University of Paris. From 1964-67, he was an associate professor at the university and from 1967-1973 he was a full professor. His lecture notes were the basis of the popular textbook, Mécanique quantique, which he wrote with two of his colleagues. He also continued his research work on atom-photon interactions, and his research team developed the model of the dressed atom.

In 1973, he became a professor at the Collège de France. In the early 1980s, he started to lecture on radiative forces on atoms in laser light fields. He also formed a laboratory there with Alain Aspect, Christophe Salomon, and Jean Dalibard to study laser cooling and trapping. He even took a statistical approach to laser cooling with the use of stable distributions.

His work there eventually led to the Nobel Prize in physics in 1997 "for the development of methods to cool and trap atoms with laser light", shared with Steven Chu and william Daniel Phillips. Cohen-Tannoudji was the first physics Nobel prize winner born in an Arab country.

In 2015, Cohen-Tannoudji signed the Mainau Declaration 2015 on Climate Change on the final day of the 65th Lindau Nobel Laureate Meeting. The declaration was signed by a total of 76 Nobel Laureates and handed to then-President of the French Republic, François Hollande, as part of the successful COP21 climate summit in Paris.