In the early 1960s, Langone began his career at a Wall Street financial services company named R.W. Pressprich, where he helped develop new Business. In 1968 Langone met and persuaded Ross Perot into letting Pressprich handle Electronic Data Systems's IPO. In 1969, Langone would be named Pressprich’s President.

In 1974, Langone left to form a new firm, named Invemed – a venture capital firm. Langone began to study the home improvement Business and eventually bought stock in Handy Dan, a home improvement chain. This led to a relationship with Handy Dan CEO Bernard Marcus and CFO Arthur Blank. Although a minority shareholder, Langone effectively protected Marcus from issues that arose between Marcus and Sanford Sigiloff, the CEO of The Daylin Corporation, Handy Dan's parent company. Marcus, however, felt that if Langone sold his interest in Handy Dan, it might actually improve his relationship with Sigiloff. Shortly after Langone sold his Handy Dan stock both Marcus and Blank were fired. Langone organized financing for Marcus and Blank to found Home Depot. Now a national chain with over 300,000 employees, it is Langone's most notable Business venture.

He was elected as Director of Yum! Brands effective October 7, 1997, and is a member of the Audit Committee.

Langone served on the board of General Electric in 2001.

Universities Langone has contributed towards include Bucknell University, and NYU. Bucknell University has benefited from his $11 million donation, which financed a new Athletics and Recreation Center. Additionally, he has donated $6.5 million to NYU's Stern School to endow the Kenneth G. Langone Part-time Evening MBA program. In 2008, Kenneth and Elaine Langone made an unrestricted $200 million gift—the largest in the Medical Center's history—and the NYU Medical Center was subsequently renamed the NYU Elaine A. and Kenneth G. Langone Medical Center. Langone serves as the Chair of the Board of Trustees of the NYU Langone Medical Center.

On July 8, 2010, Geeknet announced that Kenneth Langone was elected Chairman of its Board of Directors and is now serving as CEO.

Through Home Depot, Kenneth Langone started the charitable organization Ken's Kids, which provides job training and employment to youths aged 18–21 with disabilities in Philadelphia. Langone himself financially contributes to the charity.

Langone is portrayed by actor Ray Iannicelli in the 2017 HBO Films production The Wizard of Lies. In the film, Langone is depicted as one of the few people who see through the schemes of convicted Wall Street swindler Bernie Madoff. In a scene featuring Iannicelli and Robert DeNiro as Madoff, Langone forcefully rejects an offer to invest money into a new investment fund that later turns out to be fraudulent.