Ahn was born in Highland Park, Los Angeles, California. His parents emigrated to the United States in 1902.

Ahn graduated from high school in 1923, and went to work in the rice fields around Colusa, California. The land was owned by the Hung Sa Dan, or Young Korean Academy, a Korean independence movement that trained Koreans to become Leaders of their country once it was free from Japanese rule. Since Koreans could not own land in California, the Academy put the property in Ahn's name. Unfortunately, the rice crops failed because of heavy rains, and Ahn found himself deeply in debt. He went to work as an elevator operator in Los Angeles to pay back the debt and help support his family.

He worked to have his father and mother buried together in Seoul. His father had been buried far from the city because the Japanese hoped to play down his independence work. His mother had died in California. They had not seen each other from the time Dosan returned to Korea in 1926, before the birth of his youngest son. Working with the Korean government, Ahn helped to establish a park to honor his father and was able to have his parents buried there.

It was not until 1934 that he could afford to attend the University of Southern California. His father told him if he really wanted to be an actor, he had to be the best actor he could and convinced him to take acting and cinematography courses. While still a student, he appeared in a stage production of Merrily We Roll Along, which toured the western United States.

Ahn's first film was A Scream in the Night in 1935. He appeared in the Bing Crosby film Anything Goes, though Director Lewis Milestone had initially rejected him because his English was too good for the part. His first credited roles came in 1936 in The General Died at Dawn and Stowaway, opposite Shirley Temple. He starred opposite Anna May Wong in Daughter of Shanghai (1937) and King of Chinatown (1937).

In the 1950s, Ahn opened a Chinese restaurant with his sister, Soorah. "Phil Ahn's Moongate Restaurant" was one of the first Chinese restaurants in Panorama City in the San Fernando Valley, and lasted for more than thirty years before finally closing.

In 1952, Ahn made his television debut on the Schlitz Playhouse, a series he would make three additional appearances on. Ahn would also be cast in four episodes of ABC's Adventures in Paradise, four episodes of the ABC/Warner Brothers crime drama Hawaiian Eye, and the CBS crime drama Hawaii Five-O. He made three appearances each on Crossroads, Bonanza, and M*A*S*H. He would also appear in two television movies.

Ahn appeared in Love Is a Many-Splendored Thing, Around the World in Eighty Days, Thoroughly Modern Millie and Paradise, Hawaiian Style, with Elvis Presley. He got to play Korean characters in Korean War movies such as Battle Circus (1953) and Battle Hymn (1956).

In 1968, Ahn made a USO tour of Vietnam, visiting both American and Korean troops in South Vietnam.

Ahn died on February 28, 1978, due to complications from surgery.

In 1984, Ahn was posthumously inducted into the Hollywood Walk of Fame with a motion pictures star for his contributions to the film industry. His star is located at 6211 Hollywood Boulevard.