Jarman was born in Nashville, Tennessee. He was ten years old and in the fifth grade in Nashville when he was discovered in a nationwide talent search by MGM Studios, and was cast as the lead actor in the film The Yearling (1946).

Following coursework in pre-law at Vanderbilt University, Jarman appeared in Disney's The Great Locomotive Chase (1956), which was his final movie. After that, he served three years in the U.S. Navy, doing public relations work.

He returned to acting with a role on an episode of the television production Centennial (1978). Jarman was a special guest as a past acting award winner at both the 1998 and 2003 Academy Awards Ceremonies. Later, he moved to working behind the scenes, and had success as Executive Producer of the film, "Fillmore", about rock impresario Bill Graham. He ran the San Francisco Film Festival for 15 years (1965-1980) and was known for his in-depth retrospectives of movie stars and Directors. He served as Director of Cultural Affairs for the City of San Francisco. He founded Jarman Travel Inc. in 1986 to serve the travel needs of corporations and executives. He has seven children by 3 wives, including two daughters with his current wife, Katharine. In 2018, he wrote a book about his life titled "My Life and the Final Days of Hollywood."