Sarmiento's career began during the 1950s, building residential and commercial developments. After making his name in the construction Business, Sarmiento initiated an acquisitions campaign that has persisted for decades, concentrating on banks and financial Service companies. By 2000, he was widely acknowledged as Colombia's top banking mogul, holding an estimated 22 percent of the local banking assets, close to the maximum allowed by law.

Sarmiento founded Grupo Aval as a holding company to gather his banking, telecommunications, and real estate interests. Four major banks, as well as other financial Service corporations, form the core of the organization. He is considered one of Colombia's most prudent entrepreneurs, known for his conservative management. This cautious approach helped his financial empire survive the worst recession in Colombia's history. In 2000, plans to list Grupo Aval on the New York Stock Exchange were postponed due to Wall Street's unfavorable climate for emerging market funds. Sarmiento's net worth increased significantly from 2005 to 2006, thanks largely to Colombia's booming stock market, Bolsa de Valores de Colombia, the world's second best performing in 2005. As of 2006, he continues to run Grupo Aval with help from his son Luis Carlos Jr., whom he is grooming to eventually take over.