The Jays were a prominent merchant family in New York City, descended from Huguenots who had come to New York to escape religious persecution in France. In 1685 the Edict of Nantes had been revoked, thereby abolishing the rights of Protestants and confiscating their property. Among those affected was Jay's paternal grandfather, Augustus Jay. He moved from France with his sister Saint Jay to the Virginia Colonies and then New York, where he built a successful merchant empire. Jay's father, Peter Jay, born in New York City in 1704, became a wealthy trader in furs, wheat, timber, and other commodities.

Jay's mother was Mary Van Cortlandt, who had married Peter Jay in 1728, in the Dutch Church. They had ten children together, seven of whom survived into adulthood. Mary's father, Jacobus Van Cortlandt, had been born in New Amsterdam in 1658. Cortlandt served on the New York Assembly, was twice mayor of New York City, and also held a variety of judicial and military offices. Two of his children (the other one being his son Frederick) married into the Jay family.

Jay was born on December 23, 1745 (following the Gregorian calendar), in New York City; only three months later the family moved to Rye, New York, when Peter Jay retired from Business following a smallpox epidemic that had blinded two of his children.

Jay spent his childhood in Rye. He was educated there by his mother until he was eight years old, when he was sent to New Rochelle to study under Anglican priest Pierre Stoupe. In 1756, after three years, he would return to homeschooling in Rye under the tutelage of his mother and George Murray.

From the age of three months old until he attended Kings College in 1760, Jay was raised in Rye, on a farm acquired by his father Peter in 1745 that overlooked Long Island Sound. After negotiating the Treaty of Paris that ended the Revolutionary War, Jay returned to his childhood home to celebrate with his family and friends in July, 1784. Jay inherited this property upon the death of his older brother Peter in 1813 after Jay had already established himself at Katonah. He conveyed the Rye property to his eldest son, Peter Augustus Jay, in 1822. This property remained in the Jay family through 1904.

In 1768, after reading law and being admitted to the bar of New York, Jay, with the money from the government, established a legal practice and worked there until he created his own law office in 1771. He was a member of the New York Committee of Correspondence in 1774 and became its secretary, which was his first public role in the revolution.

Jay was a slaveholder, as were many wealthy New Yorkers during the time period. However, in 1774 Jay drafted the Address to the People of Great Britain, which draws upon the image of slavery and compares the British treatment of blacks to the British treatment of all the colonists. Such comparisons between British treatment of blacks and of the colonists was Common.

In 1774, at the close of the Continental Congress, Jay returned to New York. There he served on New York City's Committee of Sixty, where he attempted to enforce a non-importation agreement passed by the First Continental Congress. Jay was elected to the third New York Provincial Congress, where he drafted the Constitution of New York, 1777; his duties as a New York Congressman prevented him from voting on or signing the Declaration of Independence. Jay served on the committee to detect and defeat conspiracies, which monitored British Actions. New York's Provincial Congress elected Jay the Chief Justice of the New York Supreme Court of Judicature on May 8, 1777, which he served on for two years.

The Continental Congress turned to Jay, a political adversary of the previous President Henry Laurens, only three days after Jay became a delegate and elected him President of the Continental Congress. In previous congresses, Jay had moved from a position of seeking conciliation with Britain to advocating separation sooner than Laurens. Eight states voted for Jay and four for Laurens. Jay served as President of the Continental Congress from December 10, 1778, to September 28, 1779. It was a largely ceremonial position without real power, and indicated the resolve of the majority and the commitment of the Continental Congress.

On September 27, 1779, Jay was appointed Minister to Spain. His mission was to get financial aid, commercial treaties and recognition of American independence. The royal court of Spain refused to officially receive Jay as the Minister of the United States, as it refused to recognize American Independence until 1783, fearing that such recognition could spark revolution in their own colonies. Jay, however, convinced Spain to loan $170,000 to the US government. He departed Spain on May 20, 1782.

On June 23, 1782, Jay reached Paris, where negotiations to end the American Revolutionary War would take place. Benjamin Franklin was the most experienced diplomat of the group, and thus Jay wished to lodge near him, in order to learn from him. The United States agreed to negotiate with Britain separately, then with France. In July 1782, the Earl of Shelburne offered the Americans independence, but Jay rejected the offer on the grounds that it did not recognize American independence during the negotiations; Jay's dissent halted negotiations until the fall. The final treaty dictated that the United States would have Newfoundland fishing rights, Britain would acknowledge the United States as independent and would withdraw its troops in exchange for the United States ending the seizure of Loyalist property and honoring private debts. The treaty granted the United States independence, but left many border regions in dispute, and many of its provisions were not enforced. John Adams credited Jay with having the central role in the negotiations noting he was “of more importance than any of the rest of us.”

Jay was a member of the Church of England, and later of the Protestant Episcopal Church in America after the American Revolution. Since 1785, Jay had been a warden of Trinity Church, New York. As Congress's Secretary for Foreign Affairs, he supported the proposal after the Revolution that the Archbishop of Canterbury approve the ordination of bishops for the Episcopal Church in the United States. He argued unsuccessfully in the provincial convention for a prohibition against Catholics holding office.

Federalist No. 2 is an article written by John Jay as the second essay of The Federalist Papers, a series of 85 essays arguing for the ratification of the United States Constitution. These essays, written by Jay, Alexander Hamilton, and James Madison, were published under the pseudonym "Publius". Federalist No. 2, titled "Concerning Dangers From Foreign Force and Influence", was published on October 31, 1787, as the first of five essays written by Jay.

A proponent of strong, centralized government, Jay worked to ratify the United States Constitution in New York in 1788. He was a co-author of "The Federalist Papers" along with Alexander Hamilton and James Madison, and wrote five of the 85 essays. After the establishment of the new federal government, Jay was appointed by President George Washington as the first Chief Justice of the United States, serving from 1789 to 1795. The Jay Court experienced a light workload, deciding just four cases over six years. In 1794, while serving as Chief Justice, Jay negotiated the highly controversial Jay Treaty with Britain. Jay received a handful of electoral votes in three of the first four presidential elections, but never undertook a serious bid for the presidency.

In September 1789, George Washington offered him the position of Secretary of State (which, though technically a new position, would have continued Jay's Service as Secretary of Foreign Affairs); he declined. Washington responded by offering him the new title—which Washington stated "must be regarded as the keystone of our political fabric"—as Chief Justice of the United States, which Jay accepted. Washington officially nominated Jay on September 24, 1789, the same day he signed the Judiciary Act of 1789 (which created the position of Chief Justice) into law. Jay was unanimously confirmed by the United States Senate on September 26, 1789; Washington signed and sealed Jay's commission the same day. Jay swore his oath of office on October 19, 1789. Washington also nominated John Rutledge, william Cushing, Robert Harrison, James Wilson, and John Blair Jr. as Associate Judges. Harrison declined the appointment, however, and Washington appointed James Iredell to fill the final seat on the Court. Jay would later serve with Thomas Johnson, who took Rutledge's seat, and william Paterson, who took Johnson's seat. While Chief Justice, Jay was elected a Fellow of the American Academy of Arts and Sciences in 1790.

The Court's Business through its first three years primarily involved the establishment of rules and procedure; reading of commissions and admission of attorneys to the bar; and the Justices' duties in "riding circuit," or presiding over cases in the circuit courts of the various federal judicial districts. No convention existed that precluded the involvement of Supreme Court Justices in political affairs, and Jay used his light workload as a Justice to freely participate in the Business of Washington's administration. He used his circuit riding to spread word throughout the states of Washington's commitment to neutrality, then published reports of French minister Edmond-Charles Genet's campaign to win American support for France. However, Jay also established an early precedent for the Court's independence in 1790, when Treasury Secretary Alexander Hamilton wrote to Jay requesting the Court's endorsement of legislation that would assume the debts of the states. Jay replied that the Court's Business was restricted to ruling on the constitutionality of cases being tried before it and refused to allow it to take a position either for or against the legislation.

Its first case did not occur until early in the Court's third term, with West v. Barnes (1791). The Court had an early opportunity to establish the principle of judicial review in the United States with the case, which involved a Rhode Island state statute permitting the lodging of a debt payment in paper currency. Instead of grappling with the constitutionality of the law, however, the Court unanimously decided the case on procedural grounds, strictly interpreting statutory requirements.

In 1792, Jay was the Federalist candidate for governor of New York, but he was defeated by Democratic-Republican George Clinton. Jay received more votes than George Clinton; but, on technicalities, the votes of Otsego, Tioga and Clinton counties were disqualified and, therefore, not counted, giving George Clinton a slight plurality. The State constitution said that the cast votes shall be delivered to the secretary of state "by the sheriff or his deputy"; but, for Example, the Otsego County Sheriff's term had expired, so that legally, at the time of the election, the office of Sheriff was vacant and the votes could not be brought to the State capital. Clinton partisans in the State legislature, the State courts, and Federal offices were determined not to accept any argument that this would, in practice, violate the constitutional right to vote of the voters in these counties. Consequently, these votes were disqualified.

In Chisholm v. Georgia (1793), the Jay Court had to answer the question: "Was the state of Georgia subject to the jurisdiction of the Supreme Court and the federal government?" In a 4–1 ruling (Iredell dissented and Rutledge did not participate), the Jay Court ruled in favor of two South Carolinan Loyalists who had had their land seized by Georgia. This ruling sparked debate, as it implied that old debts must be paid to Loyalists. The ruling was overturned when the Eleventh Amendment was ratified, as it ruled that the judiciary could not rule on cases where a state was being sued by a citizen of another state or foreign country. The case was brought again to the Supreme Court in Georgia v. Brailsford, and the Court reversed its decision. However, Jay's original Chisholm decision established that states were subject to judicial review.

Relations with Britain verged on war in 1794. British exports dominated the U.S. market, while American exports were blocked by British trade restrictions and tariffs. Britain still occupied northern forts that it had agreed to surrender in the Treaty of Paris. Britain’s impressment of American sailors and seizure of naval and military supplies bound to enemy ports on neutral ships also created conflict. Madison proposed a trade war, "A direct system of commercial hostility with Great Britain," assuming that Britain was so weakened by its war with France that it would agree to American terms and not declare war. Washington rejected that policy and sent Jay as a special envoy to Great Britain to negotiate a new treaty; Jay remained Chief Justice. Washington had Alexander Hamilton write instructions for Jay that were to guide him in the negotiations. In March 1795, the resulting treaty, known as the Jay Treaty, was brought to Philadelphia. When Hamilton, in an attempt to maintain good relations, informed Britain that the United States would not join the Danish and Swedish governments to defend their neutral status, Jay lost most of his leverage. The treaty eliminated Britain's control of northwestern posts and granted the United States "most favored nation" status, and the U.S. agreed to restricted commercial access to the British West Indies.

While in Britain, Jay was elected in May 1795, as the second governor of New York (succeeding George Clinton) as a Federalist. He resigned from the Supreme Court Service on June 29, 1795, and served six years as governor until 1801.

While governor, Jay ran in the 1796 presidential election, winning five electoral votes, and in the 1800 election, winning one vote.

In 1801, Jay declined both the Federalist renomination for governor and a Senate-confirmed nomination to resume his former office as Chief Justice of the United States, retiring to the life of a farmer in Westchester County, New York. Soon after his retirement, his wife died. Jay remained in good health, continued to farm and, with one notable exception, stayed out of politics. In 1819, he wrote a letter condemning Missouri's bid for admission to the union as a slave state, saying that slavery "ought not to be introduced nor permitted in any of the new states".

The treaty did not resolve American grievances about neutral shipping rights and impressment, and the Democratic-Republicans denounced it, but Jay, as Chief Justice, decided not to take part in the debates. The continued British impressment of American ships would lead, in part, to the War of 1812. The failure to get compensation for slaves taken by the British during the Revolution "was a major reason for the bitter Southern opposition". Jefferson and Madison, fearing a commercial alliance with aristocratic Britain might undercut republicanism, led the opposition. However, Washington put his prestige behind the treaty and Hamilton and the Federalists mobilized public opinion. The Senate ratified the treaty by a 20–10 vote (just enough to meet the two-thirds majority requirement). Democratic-Republicans were incensed at what they perceived as a betrayal of American interests, and Jay was denounced by protesters with such graffiti as "Damn John Jay! Damn everyone who won't damn John Jay!! Damn everyone that won't put Lights in his windows and sit up all night damning John Jay!!!" One newspaper Editor wrote, "John Jay, ah! the arch traitor – seize him, drown him, burn him, flay him alive." Jay himself quipped that he could travel at night from Boston to Philadelphia solely by the light of his burning effigies.

Midway through Jay's retirement in 1814, both he and his son Peter Augustus Jay were elected members of the American Antiquarian Society.

Jay, who served as vice-president (1816–21) and President (1821–27) of the American Bible Society, believed that the most effective way of ensuring world peace was through propagation of the Christian gospel. In a letter addressed to Pennsylvania House of Representatives member John Murray, dated October 12, 1816, Jay wrote, "Real Christians will abstain from violating the rights of others, and therefore will not provoke war. Almost all nations have peace or war at the will and pleasure of rulers whom they do not elect, and who are not always wise or virtuous. Providence has given to our people the choice of their rulers, and it is the duty, as well as the privilege and interest, of our Christian nation to select and prefer Christians for their rulers." He also expressed a belief that the moral precepts of Christianity were necessary for good government, saying, "No human society has ever been able to maintain both order and freedom, both cohesiveness and liberty apart from the moral precepts of the Christian Religion. Should our Republic ever forget this fundamental precept of governance, we will then, be surely doomed."

The Society helped enact the 1799 law for gradual emancipation of slaves in New York, which Jay signed into law as governor. "An Act for the Gradual Abolition of Slavery" provided that, from July 4 of that year, all children born to slave parents would be free (subject to lengthy apprenticeships) and that slave exports would be prohibited. These same children would be required to serve the mother’s owner until age 28 for males and age 25 for females, years beyond the typical period of indenture. The law thus defined a type of indentured servant while providing for eventual freedom for children born to slaves. It did not provide government payment of compensation to the owners. It also provided legal protection and assistance for free blacks kidnapped for the purposes of being sold into slavery. All slaves were emancipated by July 4, 1827. The process in New York may perhaps have been the largest total emancipation in North America between 1783 and 1861.

On the night of May 14, 1829, Jay was stricken with palsy, probably caused by a stroke. He lived for three days, dying in Bedford, New York, on May 17. Jay had chosen to be buried in Rye, where he lived as a boy. In 1807, he had transferred the remains of his wife Sarah Livingston and those of his colonial ancestors from the family vault in the Bowery in Manhattan to Rye, establishing a private cemetery. Today, the Jay Cemetery is an integral part of the Boston Post Road Historic District, adjacent to the historic Jay Estate. The Cemetery is maintained by the Jay descendants and closed to the public. It is the oldest active cemetery associated with a figure from the American Revolution.

What remains of the original 400-acre (1.6 km) property is a 23-acre (93,000 m) parcel called the Jay Estate. In the center rises the 1838 Peter Augustus Jay House, built by Peter Augustus Jay over the footprint of his father's ancestral home, "The Locusts"; pieces of the original 18th century farmhouse were incorporated into the 19th century structure. Stewardship of the site and several of its buildings for educational use was entrusted in 1990 by the New York State Board of Regents to the Jay Heritage Center. In 2013, the non-profit Jay Heritage Center was also awarded stewardship and management of the site's landscape which includes a meadow and gardens.

On September 5, 1936, the Rye Post Office issued a special cancellation stamp in honor of their native son. To further commemorate the fact that Rye was Jay's hometown, the Rye Post office United States Post Office led by Congresswoman Caroline Love Goodwin O'Day commissioned Painter Guy Pene du Bois to create a mural for the post office's lobby, titled "John Jay at His Home." It was completed in 1938 during the WPA era. On December 12, 1958, the United States Postal Service released a 15¢ Liberty Issue postage stamp honoring Jay.

High schools named after Jay are located in Brooklyn, NY, Cross River and Hopewell Junction, New York and San Antonio, Texas. Exceptional undergraduates at Columbia University are designated John Jay Scholars, and one of that university's undergraduate dormitories is known as John Jay Hall. In 1964, the City University of New York's College of Police Science was officially renamed the John Jay College of Criminal Justice.