A native of Kunshan County in Suzhou Prefecture, he was born in Shanghai, China, His Father taught English at an elementary school outside Shanghai, while his mother Zen Wan (Chien) Wang was a homemaker. He graduated from Shanghai Jiao Tong University with a degree in electrical engineering in 1940. He emigrated to the United States in June 1945 to attend Harvard University for graduate school, earning a PhD in applied physics in 1948. After graduation, he worked at Harvard with Howard Aiken on the design of the Mark IV, Aiken's first fully electronic computer. Wang co-invented the pulse transfer controlling device with Way-Dong Woo, a schoolmate from China who fell ill before their patent was issued. The new device implemented write-after-read which made magnetic core memory possible. Harvard reduced its commitment to computer research in 1951, prompting Wang to start his own engineering Business.

Wang founded Wang Laboratories in June 1951 as a sole proprietorship. The first years were lean and Wang raised $50,000 working capital by selling one third of the company to a machine tools manufacturer Warner & Swasey Company. In 1955 when the core memory patent was issued, Wang sold it to IBM for $500,000 and incorporated Wang Laboratories with Ge-Yao Chu, a schoolmate. The company grew slowly and in 1964 sales reached $1,000,000. Wang began making desktop electronic calculators with digital displays, including a centralised calculator with remote terminals for group use.

By 1970 the company had sales of $27 million and 1,400 employees. They began Manufacturing word processors in 1974, copying the already popular Xerox Redactron word processor, a single-user, cassette-based product. In 1976 they started marketing a multi-user, display-based product, based on the Zilog Z80 processor. Typical installations had a master unit (supplying disk storage) connected to intelligent diskless slaves which the operators used. Connections were via dual coax using differential signaling in an 11-bit asynchronous ASCII format clocked at 4.275 MHz. This product became the market leader in the word processing industry. In addition to calculators and word processors, Wang's company Diversified into minicomputers in the early 1970s. The Wang 2200 was one of the first desktop computers with a large CRT display and ran a fast hardwired BASIC interpreter. The Wang VS system was a multiuser minicomputer whose instruction set was very close to the design of IBM's System/370. It was not binary-compatible because register usage conventions and system call interfaces were different. The Wang VS serial terminals could be used in data processing mode and word processing mode. They were user-programmable in data-processing mode and used the same word processing software as the earlier dedicated word processing systems.

An Wang also made a substantial contribution for the restoration of a Boston landmark, which was then called the Metropolitan Theatre. The "Met" was renamed in 1983 as The Wang Theatre, and the Metropolitan Center became known as the Wang Center for the Performing Arts. Wang donated $4 million to Massachusetts General Hospital's ambulatory care center, which was renamed to the Wang Building.

An Wang also founded the Wang Institute of Graduate Studies in Tyngsborough, Massachusetts which offered a graduate program in software engineering. He made substantial donations to this organization, including the proceeds of his autobiography, Lessons. However, enrollment remained low, and in 1987, after nearly a decade of operation, Wang decided to discontinue funding the institution and transferred ownership of the campus to Boston University.

He was inducted into the National Inventors Hall of Fame in 1988. The Dr. An Wang Middle School in Lowell, Massachusetts, is named in his honor, as is the An Wang Professorship of Computer Science and Electrical Engineering at Harvard University, held by Roger W. Brockett and Hanspeter Pfister.

Wang Laboratories, which in 1989 once employed over 30,000 people, was headquartered in Tewksbury, Massachusetts and later Lowell, Massachusetts. When Wang looked to retire from actively running his company in 1986, he insisted upon handing over the corporate reins to his son Fred Wang. Hard times ensued for the company and the elder Wang was eventually forced to remove his son in 1989.

An Wang died of cancer in 1990. He and his wife, Lorraine (Chur) Wang, lived in Lincoln, Massachusetts. Lorraine Wang died on March 1, 2016, at Emerson Hospital in Concord, Massachusetts. Their three children are: