Hill chose to build his railroad north of the competing Northern Pacific line, which had reached the Pacific North West over much more difficult terrain with more bridges, steeper grades, and tunnelling. Hill did much of the route planning himself, travelling over proposed routes on horseback. The key to the Great Northern line was Hill's use of the previously unmapped Marias Pass. The pass had initially been described by Lewis and Clark in 1805, but no one since had been able to find it so Hill hired Santiago Jameson to search it out. Jameson discovered the pass 1889 and it shortened the Great Northern's route by almost one hundred miles. The pass had been discovered by John Frank Stevens, principal Engineer of the Great Northern Railway, in December 1889, and offered an easier route across the Rockies than that taken by the Northern Pacific. The Great Northern reached Seattle in 1893.

Hill was born in Eramosa Township, Wellington County, Upper Canada (now Ontario). A childhood accident with a bow and arrow blinded him in the right eye. He had nine years of formal schooling. He attended the Rockwood Academy for a short while, where the head gave him free tuition. He was forced to leave school in 1852 due to the death of his Father. By the time he had finished, he was adept at algebra, geometry, land surveying, and English. His particular talents for English and mathematics would be critical later in his life.

After working as a clerk in Kentucky (during which he learned bookkeeping), Hill decided to permanently move to the United States and settled in St. Paul, Minnesota, at the age of 18. His first job in St. Paul was with a steamboat company, where he worked as a bookkeeper. By 1860, he was working for wholesale grocers, for whom he handled freight transfers, especially dealing with railroads and steamboats. Through this work, he learned all aspects of the freight and transportation Business. During this period, Hill began to work for himself for the first time. During the winter months when the Mississippi River was frozen and steamboats could not run, Hill started bidding on other contracts and won quite a few.

In 1867, James J. Hill married Mary Theresa Mehegan, born in 1846 in New York City. They had ten children:

Because of his previous experiences in shipping and fuel supply, Hill was able to enter both the coal and steamboat businesses. In 1870, he and his partners started the Red River Transportation Company, which offered steam boat transportation between St. Paul and Winnipeg. By 1879 he had a local monopoly by merging (with Norman Kittson). In 1867, Hill entered the coal Business, and by 1879 it had expanded five times over, giving Hill a local monopoly in the anthracite coal Business. During this same period, Hill also entered into banking and quickly managed to become member of several major banks' boards of Directors. He also bought out bankrupt businesses, built them up again, and then resold them—often gaining a substantial profit. Hill noted that the secret to his success was "work, hard work, intelligent work, and then more work."

During the Panic of 1873, a number of railroads, including the St. Paul and Pacific Railroad (StP&P), had gone bankrupt. The StP&P in particular was caught in an almost hopeless legal muddle. For James Hill it was a golden opportunity. For three years, Hill researched the StP&P and finally concluded that it would be possible to make a good deal of money off the StP&P, provided that the initial capital could be found. Hill teamed up with Norman Kittson (the man he had merged steamboat businesses with), Donald Smith, George Stephen and John Stewart Kennedy. Together they not only bought the railroad, they also vastly expanded it by bargaining for trackage rights with the Northern Pacific Railway. In May 1879, the St. Paul, Minneapolis, and Manitoba Railway Co. (StPM&M) formed—with James J. Hill as general manager. His first goal was to expand and upgrade even more.

Hill was a hands-on, detail-obsessed manager. A Canadian himself of Scotch-Irish Protestant ancestry, he brought in many men with the same background into high management. He wanted people to settle along his rail lines, so he sold homesteads to immigrants while transporting them to their new homes using his rail lines. When he was looking for the best path for one of his tracks to take, he went on horseback and scouted it personally. Under his management, StPM&M prospered. In 1880, its net worth was $728,000; in 1885 it was $25,000,000.

Between 1883 and 1889, Hill built his railroads across Minnesota, into Wisconsin, and across North Dakota to Montana.

In this time he also began to focus his energies on securing trade with Asian countries. He offered Japanese Industrialists Southern cotton and ship it free if they would compare it with the short staple cotton they were using with the promise of a refund if they were dissatisfied, which they were not. With these friendly relations established Hill managed to secure the industrializing Japanese order for 15,000 tons of rails against competition from England and Belgium. From 1886 to 1905 American exports to Japan leapt from $7.7 million a year to $51.7 million.

In 1887, the Great Northern's first company headquarters building was constructed in St. Paul. It was designed by James Brodie, who also built the Hill's house on Summit Avenue. The 1887 building was converted between 2000 and 2004 to a 53 unit condo in the Historic Lowertown District of St. Paul. Hill had seen the devastation done downtown by the Great Chicago Fire. As a result, one feature Hill integrated into the construction of the 1887 company headquarters (the Great Northern General Office Building) was barrel vaulted ceilings constructed of brick and railroad steel rails that held up a layer of sand several inches deep. The theory was that if a fire broke out and the ceiling caved in, the sand would drop and retard or suppress the fire.

When there was not enough industry in the areas Hill was building, Hill brought the industry in, often by buying out a company and placing plants along his railroad lines. By 1889, Hill decided that his Future lay in expanding into a transcontinental railroad.

In 1891, after three years of building, construction was completed on a new Hill family home on Summit Avenue in St. Paul. Over 400 workers labored on the project. Built at a cost of $930,000 and with 36,000 square feet (3,300 m), the James J. Hill House was among the city's largest. As with his Business dealings, Hill supervised the construction and design himself, hiring and firing several Architects in the process. The house has many early electrical and mechanical systems that predate widespread adoption in modern domestic structures. Upon completion of the Summit Avenue residence, Hill had the family's old house, which he had constructed in 1878, razed. After the death of Hill's wife in 1921, the house was donated to the Roman Catholic Archdiocese of Saint Paul and Minneapolis. It was obtained by the Minnesota Historical Society in 1978 and today is operated as a museum and gallery.

In 1893, Hill began the process of looking for a source of labor other than Chinese workers. For a brief period of time, he hired Italian and Greek laborers, but company officials were not satisfied with their performance. Hill sent emissaries to the Pacific who found that Japan had the most potential in the market of "Oriental Trade," and he decided to capitalize on this opportunity.

Politically, Hill was a Bourbon Democrat. The Democratic Party's continued enchantment with the populist william Jennings Bryan led Hill to support Republican presidential candidates william McKinley (1896 and 1900), Theodore Roosevelt (1904), and william Howard Taft (1908 and 1912).

In 1898 Hill purchased control of large parts of the Mesabi Range iron mining district in Minnesota, along with its rail lines. The Great Northern began large-scale shipment of ore to the steel mills of the Midwest.

Leonard says that after 1900 Hill exhibited poor Business judgment regarding one Canadian subsidiary, the Vancouver, Westminster and Yukon Railway Company (VW&Y). He ousted its President John Hendry, thereby worsening the problems, prolonging the delays, and adding to the costs of taking over the VW&Y. Hill's top aides were careless about details, bookkeeping, correspondence, and reports.

With 1901 and the start of the new century, James Hill now had control of both the Great Northern Railway, and the Northern Pacific (which he had obtained with the help of his friend J. P. Morgan, when that railroad went bankrupt in the depression of the mid-1890s). Hill also wanted control of the Chicago, Burlington and Quincy railroad because of its Midwestern lines and access to Chicago. The Union Pacific Railroad was the biggest competitor of Great Northern and Northern Pacific Railroads. Although Great Northern and Northern Pacific were backed by J. P. Morgan and James J. Hill, the Union Pacific was backed not only by its President, Edward H. Harriman, but by the extremely powerful william Rockefeller and Jacob Schiff.

Roosevelt sent his Justice Department to sue the Northern Securities Company in 1902. The Supreme Court in 1904 ordered it to be dissolved as a monopoly. (Ironically, the Burlington Route, Northern Pacific, and Great Northern would later merge in 1970 to form the Burlington Northern Railroad.) This unfortunately ended Hill's ability to maintain competitive rates in Asian countries and in the subsequent two years American trade with Japan and China dropped 40% (or $41 million). Hill moved on without the benefit of a central company, and acquired the Colorado and Southern Railway lines into Texas. He also built the Spokane, Portland and Seattle Railway. By the time of his death in 1916, James J. Hill was worth more than $53 million (almost $2.5 billion). When his estate was divided his widow received over 16 million, and each of his children received almost 4 million; only 1.5 million was paid in income and inheritance taxes.

Drawing on his experience in the development of Minnesota's Iron Range, Hill was, during 1911–1912, in close contact with Gaspard Farrer of Baring Brothers & Company of London regarding the formation of the Brazilian Iron Ore Company to tap that nation's rich mineral deposits.

Hill was intimately involved in the planning and construction (1914–1916) of a new company headquarters in St. Paul (to be known as the Great Northern Office Building), which was to house the corporate staffs of the Great Northern, the Northern Pacific and Hill's banking enterprises. The 14-story building cost $14 million to construct.

Near the end of his life, Hill played what a recent biographer, Albro Martin, called his "last and greatest role." After the first punishing year of World War I, the Allied Powers desperately needed financial support to continue the war effort. To that end, Hill was a major figure in the effort launched by J.P. Morgan to float the Anglo-French Bond drive of 1915, which allowed the Allies to purchase much-needed foodstuffs and other supplies. In September 1915, the first public loan, the $500,000,000 Anglo-French loan, was floated after negotiations with the Anglo-French Financial Commission. Concomitantly, the resulting trade in munitions with England and France carried the United States from a depression in 1914 to boom years in 1915 and 1916.

By early 1916 Hill began pouring more attention into philanthropy, donating thousands of dollars to various institutions as he privately struggled with a variety of increasingly painful ailments. His condition deteriorated quickly in mid-May, but even with the help of many respected doctors he was beyond saving. After falling into a coma, he died in his home in St. Paul, Minnesota, on May 29, 1916. Mary Hill died in 1922 and was buried next to her husband by the shore of Pleasant Lake on their North Oaks farm. Because of vandals and curious admirers, both graves were later moved to Resurrection Cemetery in St. Paul for safer keeping.

Hillsboro, North Dakota; Hill County, Montana; and Hillyard, Washington (now a neighborhood of Spokane) - are named for him. In 1929, the Great Northern Railway named its flagship Passenger train the Empire Builder in his honor. The train continues as Amtrak's daily Empire Builder, which uses former Great Northern tracks west of St. Paul, Minnesota. The James J. Hill House in St. Paul, Minnesota, is a National Historic Landmark.

In 1959, Hill High in St. Paul, Minnesota, was established as a school from the funds set aside from Hill's wife for education. The school, which was all-male, consolidated in 1971 with the all-female Archbishop Murray School to form Hill-Murray School in Maplewood, Minnesota.

The Great Northern energetically promoted settlement along its lines in North Dakota and Montana, especially by Germans and Scandinavians from Europe. The Great Northern bought its lands from the federal government—it received no land grants—and resold them to farmers at cheap prices. It operated agencies in Germany and Scandinavia that promoted its lands, and brought families over at low cost. Hill also invested in founding schools and churches for these communities and promoted a variety of progressive techniques to ensure they prospered. This "Dakota Boom" peaked in 1882 as 42,000 immigrants, largely from northern Europe, poured into the Red River Valley running through the region. The rapidly increasing settlement in North Dakota's Red River Valley along the Minnesota border between 1871 and 1890 was a major Example of large-scale "bonanza" farming.

Hill Capital is a venture capital fund established in 2016 "aligned with James J. Hill’s belief in the cooperation of the production, distribution and exchange of wealth as outlined in his writings". The Hill library owns 75 shares. As of September 2016, the fund is not yet closed.