Jason Gould was born in Roxbury, New York, to Mary More (1798–1841) and John Burr Gould (1792–1866). His maternal grandfather, Alexander T. More, was a businessman, and his great-grandfather John More was a Scottish immigrant who founded the town of Moresville, New York. Jay Gould studied at local schools and the Hobart Academy in Hobart, Delaware County, New York.

He married Helen Day Miller (1838–1889) in 1863; the couple had six children:

The Erie Railroad encountered financial troubles in the 1850s, despite receiving loans from financiers Cornelius Vanderbilt and Daniel Drew. The Erie entered receivership in 1859 and was reorganized as the Erie Railway. Jay Gould, Drew and James Fisk engaged in stock manipulations known as the Erie War, with the result that in the summer of 1868 Drew, Fisk, and Vanderbilt lost control of the Erie, while Gould became its President.

His principal was credited with getting him a job working as a bookkeeper for a blacksmith. A year later the blacksmith offered him half interest in the blacksmith shop, which he sold to his Father during the early part of 1854. Gould devoted himself to private study, emphasizing surveying and mathematics. In 1854, Gould surveyed and created maps of the Ulster County, New York, area. In 1856 he published History of Delaware County, and Border Wars of New York, which he had spent several years writing.

In 1856, Gould entered a partnership with Zadock Pratt to create a tanning Business in Pennsylvania in what would become Gouldsboro. Eventually, he bought out Pratt, who retired. In 1856, Gould entered another partnership with Charles Mortimer Leupp, a son-in-law of Gideon Lee, and one of the leading leather merchants in the United States at the time. Leupp and Gould was a successful partnership until the Panic of 1857. Leupp lost all his money, while Gould took advantage of the opportunity of the depreciation of property value and bought up former partnership properties for himself.

In 1859 Gould began speculative investing by buying stock in small railways. Gould's father-in-law Daniel S. Miller was credited with introducing the younger man to the railroad industry, when he suggested that Gould help him save his investment in the Rutland and Washington Railroad in the Panic of 1857. Gould purchased stock for 10 cents on the dollar, which left him in control of the company. Through the Civil War era, he did more speculation on railroad stocks in New York City. In 1863 he was appointed manager of the Rensselaer and Saratoga Railroad.

These speculations in gold culminated in the panic of Black Friday, on September 24, 1869, when the premium over face value on a gold Double Eagle fell from 62 percent to 35 percent. Gould made a small profit from this operation but lost it to subsequent lawsuits. The gold corner established Gould's reputation in the press as an all-powerful figure who could drive the market up and down at will.

After being forced out of the Erie Railroad, Gould started to build up a system of railroads in the midwest and west. He took control of the Union Pacific in 1873 when its stock was depressed by the Panic of 1873 and built a viable railroad that depended on shipments by local farmers and ranchers. Gould immersed himself in every operational and financial detail of the UP system. He built an encyclopedic knowledge, then acted decisively to shape its destiny. "He revised its financial structure, waged its competitive struggles, captained its political battles, revamped its administration, formulated its rate policies, and promoted the development of resources along its lines." After Gould's death, the Union Pacific slipped and declared bankruptcy during the Panic of 1893.

By 1879, Gould gained control of three more important western railroads, including the Missouri Pacific Railroad. He controlled 10,000 miles (16,000 km) of railway, about one-ninth of the length of rail in the United States at that time, and, by 1882, he had controlling interest in 15 percent of the country's trackage. Because the railroads were making profits and had control of rate setting, his wealth increased dramatically. When Gould withdrew from management of the Union Pacific in 1883 amidst political controversy over its debts to the federal government, he realized a large profit for himself. He obtained a controlling interest in the Western Union telegraph company, and, after 1881, in the elevated railways in New York City. In 1889, Gould organized the Terminal Railroad Association of St. Louis, which acquired a bottleneck in east-west railroad traffic at St. Louis; after Gould's death the government brought an antitrust suit to eliminate the bottleneck control.

Gould died of tuberculosis on December 2, 1892, and was interred in the Woodlawn Cemetery, The Bronx, New York. His fortune was conservatively estimated for tax purposes at $72 million (= $1.76 billion in 2016, adjusted for inflation), which he willed in its entirety to his family.

Gould was a member of West Presbyterian Church at 31 West 42nd Street. It later merged with Park Presbyterian to form West-Park Presbyterian.

At the time of his death, Gould was a benefactor in the reconstruction of the Reformed Church of Roxbury, New York, now known as the Jay Gould Memorial Reformed Church. It is located within the Main Street Historic District and listed on the National Register of Historic Places in 1988. The family mausoleum was designed by Francis O'Hara.