While at Harvard, Duncan co-captained the varsity basketball team and was named a first team Academic All-American. From 1987 to 1991, Duncan played professional basketball, mostly in Australia, with teams including Melbourne's Eastside Spectres, of Australia's National Basketball League. Duncan also participated in the 2012, 2013, and 2014 NBA All-Star Weekend Celebrity Games. Comedian Kevin Hart conceded the 2014 MVP of the Celebrity game to Duncan (20 points, 11 rebounds, 6 assists). The 20 points are a Celebrity Game high.

After graduating, Duncan played professional basketball for several years until 1991. In 1992, childhood friend and investment banker John W. Rogers, Jr., appointed Duncan Director of the Ariel Education Initiative, a program mentoring children at one of the city's worst-performing elementary schools and then assisting them as they proceeded further in the education system. After the school closed in 1996, Duncan and Rogers were instrumental in re-opening it as a charter school, Ariel Community Academy. In 1999, Duncan was appointed Deputy Chief of Staff for former Chicago Public Schools CEO Paul Vallas.

Mayor Richard M. Daley appointed Duncan to serve as Chief Executive Officer of the Chicago Public Schools on June 26, 2001. Opinions vary on Duncan's success as CEO; one prominent publication notes improved test scores and describes Duncan as a consensus builder, while another finds the improvements largely a myth and is troubled by the closing of neighborhood schools and their replacement by charter schools, and what it describes as schools' militarization.

Duncan was appointed U.S. Secretary of Education by President Barack Obama and confirmed by the Senate on January 20, 2009. One of Duncan's initiatives as secretary has been a $4 billion Race to the Top competition. It asks states to vie for federal education dollars by submitting proposals that include reforms such as expanding charter schools and judging teachers partly on how well their students do on standardized tests.

In March 2011, Duncan said 82 percent of the nation’s public schools could be failing by the following year under the standards of the No Child Left Behind law. The projection amounted to a startling spike from previous data, which showed that 37 percent of schools were on track to miss targets set by the law. "Four out of five schools in America would not meet their goals under [No Child Left Behind] by next year", Duncan said in his statement.

Addressing a group of school superintendents in late 2013, Arne Duncan stated that he found it "fascinating" that some of the opposition to the Common Core State Standards Initiative came from "white suburban moms who — all of a sudden — their child isn't as brilliant as they thought they were, and their school isn't as good as they thought they were." This had sparked outrage among different facets of society, ranging from the "white suburban mom who feel marginalized and misunderstood", the "non-white parent who wants to know why everyone is only now so upset", to the "non-white parent who wants Arne Duncan to know that she (or he) hates the Common Core, too".

On May 11, 2014, Duncan was a member of the 2014 USA Basketball 3x3 Men's Championship Team. The team of Duncan, Jitim Young, Thomas Darrow and Craig Moore qualified to represent the United States in Moscow, Russia in the 24-country 2014 FIBA 3x3 World Championship from June 5–8. Duncan's schedule did not permit him to attend, but the team finished in 9th place in the 24-team tournament.

On October 2, 2015, Duncan announced he would be stepping down at the close of 2015, to be succeeded by John King, Jr. The media stated his tenure had been marked by a "willingness to plunge head-on into the heated debate about the government's role in education.".

In March 2016, Duncan announced he will be joining the Palo Alto-based education group Emerson Collective as a managing partner.