Davis has spent most of his life in Fentress County, a mostly rural county in the state's coal-mining region. He graduated from Tennessee Technological University in 1966 with a degree in agriculture. Davis, who now lives in the rural Fentress County village of Pall Mall, also owns a construction Business, Diversified Construction Co., which builds homes, apartments, and offices. Davis and his wife Lynda, an elementary school Teacher, have three daughters, Larissa, Lynn and Libby, and five grandchildren.

Davis began his political career in 1978, when he was elected mayor of Byrdstown. Midway through his term as mayor, he was elected to the Tennessee House of Representatives, where he served two terms. He gave up the seat in 1984 to run for the Democratic nomination in the 6th District when Al Gore gave it up to make a successful run for the United States Senate. He narrowly lost the primary to state Democratic Party chairman Bart Gordon. Ten years later, he ran for the Democratic nomination in the 4th District after Jim Cooper gave up the seat to make an unsuccessful run for Gore's Senate seat. He lost narrowly again, this time to one of Cooper's former assistants, Jeff Whorley, who in turn lost the general election to Republican Van Hilleary. In 1996, he was elected to the Tennessee State Senate and served two terms there.

Midway through his second term in the State Senate, in 2002, Davis ran for the Democratic nomination in the 4th District when four-term Republican incumbent Van Hilleary gave up the seat to make what would ultimately be an unsuccessful run for governor. This time, he narrowly won the primary against a self-funding opponent, Fran Marcum of Tullahoma, Tennessee, who spent nearly $2 million in the race. He went on to win a hard-fought battle in the general election, narrowly defeating Tullahoma Alderman Janice Bowling, who was also Hilleary's district Director. Davis was reelected in a 2004 rematch against Bowling and faced only nominal opposition in 2006 and 2008. Although the 4th is not considered safe for either major party, its size (it stretches across two time zones and five television markets) makes it very difficult to unseat an incumbent.

In the November 2008 general election, Davis defeated Republican candidate Monty Lankford, a hospital equipment company owner. Afterwards Davis was appointed to the House Appropriations Committee and the Energy & Water Subcommittee.

Davis is a moderate Democrat by Tennessee standards, but a conservative Democrat by national ones. He opposes abortion and gun control, stances typical of most Democrats from rural areas of the state. During his first run for Congress, he vowed not to allow his Republican opponents to "outgun me, outpray me or outfamily me." In April 2009, Davis voted against the Matthew Shepard and James Byrd, Jr. Hate Crimes Prevention Act.

In the 2010 congressional race, Davis was challenged by Republican Scott DesJarlais. Also on the ballot were independents Paul H. Curtis, James Gray, Richard S. Johnson, and Gerald York. DesJarlais won 57.1% of the vote to Davis's 38.6%--the third-largest margin of defeat for a Democratic incumbent in the 2010 cycle, and the first time an incumbent had been unseated since the district's creation in 1983.

In the wake of Tennessee passing a strict voter identification law in 2011, Davis was denied the right to vote in Fentress County on Super Tuesday in March 2012. Davis had voted in the county for about fifteen years but was purged from the roll of registered voters.