He graduated from the college preparatory high school Baltimore City College, at the time an all-male school, and then from Duke University Phi Beta Kappa and magna cum laude in 1970. He earned his law degree from the University of Chicago Law School in 1973, where he was an Editor of the University of Chicago Law Review.

From 1973 to 1975, Rubenstein practiced law in New York with Paul, Weiss, Rifkind, Wharton & Garrison. From 1975 to 1976, he served as chief counsel to the U.S. Senate Judiciary Committee's Subcommittee on Constitutional Amendments. Prior to starting Carlyle in 1987 with william E. Conway, Jr. and Daniel A. D'Aniello, Rubenstein was a deputy domestic policy advisor to President Jimmy Carter and worked in private practice in Washington, D.C.

David Rubenstein lives in Bethesda, Maryland, and was married to Alice Rubenstein (née Alice Nicole Rogoff), founder of the Alaska House New York and the Alaska Native Arts Foundation and former owner of Alaska Dispatch News. They met while Rubenstein was working for the Carter Administration and were married on May 21, 1983.

According to A Pursuit of Wealth by Sicelo P. Nkambule, David Rubenstein expressed fear that the private equity boom would end in January 2006 stating: "This has been a golden age for our industry, but nothing continues to be golden forever". One month later, he stated: "Right now we're operating as if the music's not going to stop playing and the music is going to stop. I am more concerned about this than any other issue". According to Nkambule: “These concerns proved to be right as at the end of 2007 the buyout market collapsed...As leveraged loan activity came to an abrupt stop, private equity firms were unable to secure financing for their transactions.”

Rubenstein was elected to the Board of Trustees of the University of Chicago on May 31, 2007.

In May 2008 David Rubenstein stated: "But once this period is over, once the debt on the books of the banks is sold and new lending starts, I think you'll see the private equity industry coming back in what I call the Platinum Age - better than it's ever been before. I do think that the private equity industry has a great Future and that the greatest period for private equity is probably ahead of us."

In 2010, 2013 and 2016, he provided a total of $33 million to the Law School for scholarships. The gifts will fund up to 60 full-tuition scholarships in three consecutive Law School graduating classes. Approximately 10 percent of all students from the Classes of 2017, 2018, and 2019 will be Rubenstein Scholars.

In December 2011, Rubenstein donated $4.5 million to the National Zoo for its giant panda reproduction program. The panda complex was then named the David M. Rubenstein Family Giant Panda Habitat for the next five years and conservation biologists in the U.S. and China who are awarded National Zoo fellowships for their work to save pandas would be named "David M. Rubenstein Fellows." Another $4.5 million was donated in September 2015, about four weeks after a male giant panda cub was born. He also donated $10 million to the National Gallery of Art in support of refurbishment and expansion of the East Building of the National Gallery, work that was completed in September 2016. He is on the Board of Trustees of the National Gallery.

In 2012, he donated $7.5 million towards the repair of the Washington Monument.

Rubenstein served as chairman of the board of trustees of Duke University from 2013 to 2017. Rubenstein has made several gifts to Duke University. He donated $5 million to Duke's Sanford School of Public Policy in 2002, after which Rubenstein Hall was named for him. In 2009, he donated an additional $5.75 million to the school. In 2011, he also donated $13.6 million to the Duke University Libraries in support of renovating the university's special collections library, which was named the David M. Rubenstein Rare Book & Manuscript Library. In 2012, he donated $15 million to support the university's Innovation and Entrepreneurship Initiative. That same year, he gave another $10 million to support Duke Athletics. In 2013, Rubenstein donated $10 million to fund graduate fellowships and undergraduate internships at the Sanford School of Public Policy. In 2014, Rubenstein donated $1.9 million to Jewish Life at Duke to expand programming, fund building renovations and enhance the college experience for Jewish students. In 2015, Rubenstein gave $25 million towards the new $50 million, 71,000-square foot Arts Center, set to be inaugurated in 2017. In 2017, he gave Duke another $20 million to endow a scholarship for first generation, low income students.

In 2014, he provided the lead funding for a Forum to serve as the University's principal conference center.

In November 2015, he donated $20 million for the New Commons Building at the Institute for Advanced Study. The building will be named Rubenstein Commons and will feature conference space, meeting rooms, a cafe, and office space.

In October 2016, he donated $15 million to the Department Otolaryngology - Head and Neck Surgery at the John Hopkins School of Medicine to create a hearing center focused on restoring functional hearing loss.