Bechtolsheim was born at Hängeberg am Ammersee, Bavaria, the second of four children. The isolated house had no television or close neighbors, so he experimented with electronics as a child. His family moved to Rome in 1963. Five years later, in 1968, the family relocated again, to Nonnenhorn on Lake Constance in Germany. Aged 16, he designed an industrial controller for a nearby company based on the Intel 8080 which he then programmed in binary as he had no access to assemblers. Royalties from the product supported much of his education.

As an engineering student at Technical University of Munich Bechtolsheim entered the Jugend forscht contest for young researchers, and after entering for three years, won the physics prize in 1974.

He received a Fulbright Award and moved to the USA in 1975 to attend Carnegie Mellon University, where he received his master's degree in computer engineering in 1976. In 1977, he transferred to Stanford University and became a Ph.D. student in electrical engineering.

Bechtolsheim's advisor was Forest Baskett. In 1980, Vaughan Pratt also provided leadership to the SUN project. Support was provided by the Computer Science Department and DARPA. The modular computer was used for research projects such as developing the V-System, and for early Internet routers. Bechtolsheim tried to interest other companies in Manufacturing the workstations, but only got lukewarm responses.

One of the companies building computers for VLSI design was Daisy Systems, where Vinod Khosla worked at the time. Khosla had graduated a couple years earlier from the Stanford Graduate School of Business with Scott McNealy, who managed Manufacturing at Onyx Systems. The three wrote a short Business plan and quickly received funding from venture capitalists in 1982.

Sun Microsystems had its Initial Public Offering in 1986 and reached $1 billion in sales by 1988. Bechtolsheim formed a project code-named UniSun around this time in order to design a small, inexpensive desktop computer for the educational market. The result was the SPARCstation 1 (known as "campus"), the start of another line of Sun products.

In 1995, Bechtolsheim left Sun to found Granite Systems, a Gigabit Ethernet startup focused on developing high-speed network switches. In 1996, Cisco Systems acquired the firm for $220 million, with Bechtolsheim owning 60%. He became Vice President and general manager of Cisco's Gigabit Systems Business Unit, until leaving the company in December 2003 to head Kealia, Inc.

Bechtolsheim and Cheriton were two of the first Investors in Google, investing US$100,000 each in September 1998. Bechtolsheim wrote the check to "Google Inc" prior to the company even being founded. The story that says Bechtolsheim coined the name "Google" is untrue. However, he did motivate the founders to officially organize the company under that name. When he gave the check to Larry Page and Sergey Brin, Google's founders, they had not actually yet even been legally incorporated.

Bechtolsheim received a Smithsonian Leadership Award for Innovation in 1999, a Stanford Entrepreneur Company of the year award, and is a member of the National Academy of Engineering. Bechtolsheim gave the opening keynote speech at the International Supercomputing Conference in 2009.

Bechtolsheim founded Kealia in early 2001 with Stanford Professor David Cheriton, a partner in Granite Systems, to work on advanced server technologies using the Opteron processor from Advanced Micro Devices. In February 2004, Sun Microsystems announced it was acquiring Kealia in a stock swap. Due to the acquisition, Bechtolsheim returned to Sun again as senior vice President and chief Architect. Kealia hardware Technology was used in the Sun Fire X4500 storage product.

Along with Cheriton, in 2005 Bechtolsheim launched another high-speed networking company, Arastra. Arastra later changed its name to Arista Networks. Bechtolsheim left Sun Microsystems to become the Chairman and Chief Development Officer of Arista in October, 2008, but stated he still was associated with Sun in an advisory role.

He was reportedly an early investor in Claria Corporation, which ceased operating in 2008. In 2012, Bechtolsheim invested in a Stanford incubated semantic web startup Diffbot.

Bechtolsheim invested in Tapulous, the maker of music games for the Apple iPhone. Tapulous was acquired by the Walt Disney Company in 2010. He joined George T. Haber, a former colleague at Sun, to invest in wireless chip company CrestaTech in 2006 and 2008.

That same year, he was voted by IT Pros in 2012 as the person who contributed most to server innovation in the last 20 years. Two prominent Open Compute innovation Leaders will receive awards at Open Server/Open Storage Summit 2013.

From 2015-17, Bechtolsheim was an investor in the Seed, Series A, and Series B rounds of PerimeterX, an automated attack mitigation SaaS awarded by Gartner as one of its 2017 Cool Vendors.