Born in Paris to Felix Passy, a veteran of Waterloo, and Marie-Louis-Pauline Salleron, Passy's uncle was Hippolyte Passy, a cabinet minister for both Louis Philippe and Louis Napoleon. Passy studied law and practised for a short time before accepting a position as an accountant in the State Council (Conseil de Droit) from 1846 to 1849. However, under his uncle's influence he gave up this post after three years and returned to study economics. True to his republican principles, he withdrew from politics after the coup d'état of Louis Napoleon and refused to be reconciled to the Second Empire; he was therefore ineligible for any government post. He became a professional Economist in 1857, and in 1860 he began to teach political economy both in Paris and in the provinces.

His reputation was established through his Mélanges économiques (1857) essays and a lecture series given at the University of Montpellier and published as the Leçons d'économie politique. He was an advocate of free trade and adherent to the ideas of Richard Cobden. In 1877 he became a member of the French Académie des sciences morales et politiques, a component of the Institut de France; and he was a commander of the Legion of Honor. He was President of the Society of Political Economy for 70 years.

Passy directly engaged in political questions, advocating educational reform and intervening to prevent war between France and Prussia during the Luxembourg Crisis of 1867. In 1868, he helped found the Ligue internationale et permanente de la paix (International and Permanent League of Peace) to avert possible Future conflicts and became its permanent secretary. When the organization dissolved during the Franco-Prussian War, Passy helped restructure it as the Société française des amis de la paix, which in 1889 became the Société d'arbitrage entre les Nations.

In 1881, he won election to the Chamber of Deputies, where he advocated foreign policy changes, education and labor reform, including legislation on industrial accidents. He won reelection in 1886 but lost in 1889. He also supported a system of international conflict arbitration, which was inspired by Randal Cremer's resolution that established arbitration between the United States and England. In 1888, his efforts led to a meeting between British Parliamentary members and French deputies to discuss the concept of organized arbitration. The following year, the Inter-Parliamentary Union was established with Passy as one of its Presidents. He was a member of the International Bureau of Peace at Bern, Switzerland.

He was Father to Paul Passy, a prominent Linguist, founder of the International Phonetic Association (1886).

Passy's writings and speeches advocating peace were widely recognized. In 1909, he published Pour la paix, a work which chronicled the establishment of the various peace and arbitration organizations with which he was associated. From 1881 to 1902, he was professor of political economy in several colleges.

He died on June 12, 1912. A short obituary was published in The New York Times the next day.