Powell was born in Mount Morris, New York, in 1834, the son of Joseph and Mary Powell. His Father, a poor itinerant preacher, had emigrated to the U.S. from Shrewsbury, England, in 1830. His family moved westward to Jackson, Ohio, then Walworth County, Wisconsin, before settling in rural Boone County, Illinois.

As a young man he undertook a series of adventures through the Mississippi River valley. In 1855, he spent four months walking across Wisconsin. During 1856, he rowed the Mississippi from St. Anthony, Minnesota, to the sea. In 1857, he rowed down the Ohio River from Pittsburgh to the Mississippi River, traveling north to reach St. Louis. In 1858 he rowed down the Illinois River, then up the Mississippi and the Des Moines River to central Iowa. At age 25, he was elected in 1859 to the Illinois Natural History Society.

Powell studied at Illinois College, Illinois Institute (which would later become Wheaton College), and Oberlin College, over a period of seven years while teaching, but was unable to attain his degree. During his studies Powell acquired a knowledge of Ancient Greek and Latin. Powell had a restless nature and a deep interest in the natural sciences. This Desire to learn about natural sciences was against the wishes of his Father, yet Powell was still determined to do so. In 1860 when Powell was on a lecture tour he decided that the Civil War was inevitable; he decided to study military science and engineering to prepare himself for the imminent conflict.

During the Civil War, he served first with the 20th Illinois Volunteers. While stationed at Cape Girardeau, Missouri, he recruited an artillery company that became Battery "F" of the 2nd Illinois Light Artillery with Powell as captain. On November 28, 1861, Powell took a brief leave to marry the former Emma Dean. At the Battle of Shiloh, he lost most of his right arm when struck by a minie ball while in the process of giving the order to fire. The raw nerve endings in his arm would continue to cause him pain for the rest of his life.

After 1867, Powell led a series of expeditions into the Rocky Mountains and around the Green and Colorado rivers. One of these expeditions was with his students and his wife, to collect specimens all over Colorado. Powell, william Byers, and five other men were the first white men to climb Longs Peak in Colorado in 1868.

Powell retraced part of the 1869 route in 1871–1872 with another expedition that traveled the Colorado River from Green River, Wyoming to Kanab Creek in the Grand Canyon. This trip resulted in photographs (by John K. Hillers), an accurate map and various papers. At least one Powell scholar, Otis R. Marston, noted the maps produced from the survey were impressionistic rather than precise. In planning this expedition, he employed the services of Jacob Hamblin, a Mormon missionary in southern Utah and northern Arizona, who had cultivated excellent relationships with Native Americans. Before setting out, Powell used Hamblin as a negotiator to ensure the safety of his expedition from local Indian groups.

In 1874, the intellectual gatherings Powell hosted in his home were formalized as the Cosmos Club. The club has continued, with members elected to the club for their contributions to scholarship and civic activism.

In 1875, Powell published a book based on his explorations of the Colorado, originally titled Report of the Exploration of the Columbia River of the West and Its Tributaries. It was revised and reissued in 1895 as The Exploration of the Colorado River and Its Canyons.

As an ethnologist and early Anthropologist, Powell was a follower of Lewis Henry Morgan. He Classified human societies into "savagery," "barbarism" and "civilization". Powell's criteria were based on consideration of adoption of Technology, family and social organization, property relations, and intellectual development. In his view, all societies were to progress toward civilization. Powell is credited with coining the word "acculturation", first using it in an 1880 report by the U.S. Bureau of American Ethnography. In 1883, Powell defined "acculturation" as psychological changes induced by cross-cultural imitation.

In 1881, Powell was appointed the second Director of the U.S. Geological Survey, a post he held until his resignation in 1894, being replaced by Charles Walcott. He was also the Director of the Bureau of Ethnology at the Smithsonian Institution until his death. Under his leadership, the Smithsonian published an influential classification of North American Indian languages.

At an 1883 irrigation conference, Powell would prophetically remark: "Gentlemen, you are piling up a heritage of conflict and litigation over water rights, for there is not sufficient water to supply the land." Powell's recommendations for development of the West were largely ignored until after the Dust Bowl of the 1920s and 1930s, resulting in untold suffering associated with pioneer subsistence farms that failed due to insufficient rain and other.

In 1898, Powell was elected a member of the American Antiquarian Society

In the early 1900s the journals of the crew began to be published starting with Dellenbaugh's "A Canyon Voyage" in 1908, followed by the Thompson diary in 1939. Finally Bishop, Steward, W. C. Powell, and Jones' diaries were all published in 1947. These diaries made it clear Powell's writings contained some exaggerations and recounted activities that occurred on the second river trip as if they occurred on the first. They also revealed Powell used a life jacket while the other men did not have one.

Despite the loss of an arm, he returned to the Army and was present at Champion Hill, Big Black River Bridge on the Big Black River and in the siege of Vicksburg. Always the Geologist he took to studying rocks while in the trenches at Vicksburg. He was made a major and commanded an artillery brigade with the 17th Army Corps during the Atlanta Campaign. After the fall of Atlanta he was transferred to George H. Thomas' army and participated in the battle of Nashville. At the end of the war he was made a brevet lieutenant colonel, but preferred to use the title of "Major".