Joel Palmer was born in Elizabethtown, Ontario, Canada, on October 4, 1810. His parents, Hannah Phelps and Ephraim Palmer, were American Quakers. When he was two years old, they moved to New York's Catskill Mountains in response to the War of 1812. He received only three months of formal education in elementary school. In 1822, when Palmer was 12, his parents indentured him to the Haworth family for a period of four years. When he gained his freedom, he moved to Bucks County, Pennsylvania, to work on canals and bridges.

He was married to Catherine Coffee from 1830 until her death after childbirth. On October 8, 1832, Palmer became a United States citizen. Palmer married his second wife, Sarah Ann Derbyshire, in 1836, and bought land near Laurel, Indiana, in the Whitewater Valley, where he supervised a construction project for a canal. In 1843, he was elected as a Democrat to the Indiana House of Representatives for a one-year term. Representing Franklin County, he was re-elected to the legislature in 1844.

In the spring of 1845, Palmer traveled to Oregon without his family, as captain of a wagon train of 23 wagons. Stephen Meek served as the train's paid guide. Meek left the group at Fort Hall to lead some of the members on the Meek Cutoff. The remaining parts of the wagon train reached the end of the overland Oregon Trail at the Columbia River, and unwilling to wait for transport down the dangerous Cascade Rapids, Palmer's party joined Sam Barlow's party in a quest for passage through the Cascade Range around the south side of Mount Hood. Palmer climbed to the 9,000-foot level of Mount Hood on October 7, 1845—with little food and the scant protection of moccasins—to scout a route off the mountains. This was Mount Hood's first recorded climb; the Palmer Glacier on the mountain is named for him.

In 1846, Palmer returned to his family in Indiana and in 1847 he published his diary as Palmer's Journal of Travels Over the Rocky Mountains, 1845–1846. This book provided equipment guidance and comprehensive route information for those crossing the Oregon Trail. The publication also had a general description of the Oregon Country, a detailed description of the Willamette Valley, and included a copy of the Organic Laws of Oregon adopted by settlers at the Champoeg Meetings. It was a popular guidebook for immigrants for the next ten years.

Also in 1847, Palmer traveled with his family to Oregon as captain of that year's major wagon train. While passing through the Walla Walla Valley he met Marcus and Narcissa Whitman at their mission shortly before their deaths in the Whitman massacre—the event that precipitated the Cayuse War. Perhaps motivated by meeting the Whitmans, Palmer later returned to serve as a peace commissioner to tribes considering joining the Cayuse. At the outset of the war he was appointed as commissary-general of the Provisional Government’s militia forces.

After the war, in 1848, Palmer joined the California Gold Rush but returned in 1849 to co-found Dayton, Oregon on the lower Yamhill River where he built a sawmill on his donation land claim.

Palmer gained an anti-settler reputation among immigrants, newspapers and officials, who said he acted too favorably toward the Indians, even though he moved the tribes to reservations outside the Willamette Valley, seeking to avoid friction between settlers and natives by physical distance. In late 1855, while moving the Rogue River tribes to the Grand Ronde Reservation, violent resistance was threatened by settlers who felt the land should not be given to the tribes. Palmer succeeded, but the territorial legislature petitioned for his removal from office, which became effective in 1857.

After leaving office as Indian Affairs Superintendent, Palmer worked his farm on his land claim and operated his sawmill and several other enterprises. Between 1858 and 1861 he spent time in British Columbia as a merchant to prospectors in the gold rushes of the Thompson River, Similkameen Valley, and Fraser River. Palmer blazed a route to the gold fields of the Okanogan Valley and the upper portions of the Columbia River from Priest Rapids in 1860. In 1862, he was elected to the Oregon House of Representatives to represent Yamhill County. Now a member of the Republican Party, he was named Speaker of the House during that session.

That year Palmer also established the Columbia River Road Company to build a trail through the Columbia River Gorge on the Oregon side of the river. In 1864, Palmer was elected to the State Senate and served in that chamber through 1866. This included the 1865 special session of the legislature when Oregon adopted the Thirteenth Amendment to the United States Constitution that abolished slavery throughout the United States. He ran for governor in the 1870 election as the Republican candidate, but was narrowly defeated by La Fayette Grover, largely for his Indian policies.

Palmer's brother, named Ephraim like their father, also immigrated to Oregon and served as a captain in the first regiment of the Oregon Infantry. In 1871, Joel was the state's Indian agent to the Siletz tribe, remaining in the office until 1873. All eight of Palmer's children completed higher education. Joel Palmer died in Dayton on June 9, 1881, at the age of 70. His former home that he built in 1852, Palmer House, was added to the National Register of Historic Places in 1987, and is now operating as a restaurant. Palmer's name is one of 158 memorialized in the frieze of the two chambers of the Oregon Legislative Assembly at the Oregon State Capitol, with his located in the Senate chamber. During World War II the SS Joel Palmer Liberty ship was built and named in his honor.

The Palmer House, his former home in Dayton, was added to the National Register of Historic Places in 1987.