O'Herlihy was born in Wexford Town, Ireland, in 1919. His family moved to Dublin at a young age. He was educated at Christian Brothers College in Dún Laoghaire and later studied at University College Dublin, graduating in 1944 with a degree in Architecture.

His first acting role came in 1944, when he played the lead in the play Red Roses For Me written and directed by Sean O'Casey. O'Herlihy first appeared in film in Carol Reed's Odd Man Out in 1947. His first American film role was as Macduff in Orson Welles' version of Macbeth (1948). In 1952, he starred in the Red Scare film Invasion U.S.A. and, in 1954 in Luis Buñuel's Robinson Crusoe, which earned him an Academy Award nomination for Best Actor. O'Herlihy recalled that the producers of the film wanted Buñuel to use Orson Welles for the role, with Buñuel refusing saying he was too big and too fat. They arranged a screening of Welles' Macbeth to show how a bearded Welles would look but Buñuel demanded O'Herlihy who appeared in the film.

Dan O'Herlihy married Elsie Bennett in 1945. He was the brother of Director Michael O'Herlihy (1929–1997) and the father of actor Gavan O'Herlihy, visual Artist Olwen O'Herlihy, and Architect Lorcan O'Herlihy. He was a Democrat.

O'Herlihy was later featured in The Young Land in 1959 as Judge Millard Isham. In 1960, he played Sir Harry Ivers, an upper-class English drifter who joins Alan Ladd in a plot to ruin an Arizona cattle town by robbing its bank in the western One Foot in Hell. In 1964, he starred in Fail Safe in the role of General Black, or "Blackie". In 1969, he was cast in The Big Cube and 100 Rifles. In 1970, he starred in the epic Waterloo, playing the part of Michel Ney, the Marshal of France. In 1982, he starred in Halloween III: Season of the Witch and in 1984 he appeared in The Last Starfighter as Grig, Alex Rogan's reptilian copilot, navigator, and sidekick. In 1986's The Whoopee Boys he played a judge and in 1987, he was cast in John Huston's The Dead. In 1990, he appeared in RoboCop 2, the sequel to the 1987 film.

O'Herlihy had a fairly extensive career in television, having appeared in such shows as CBS's anthology series, CBS's Rawhide, as John Cord in "The incident at Dragoon Crossing", which aired in October 1960, The DuPont Show with June Allyson, on Adventures in Paradise and the crime drama, Target: The Corruptors, both on ABC. He portrayed Larry "Ace" Banner in the first season of another ABC series, The Untouchables in the episode entitled "The Big Squeeze". He was cast as Stephen Jordan in the last season of CBS's Checkmate episode " "Referendum on Murder". He appeared too on NBC's The Americans and The Man from U.N.C.L.E. in the episodes "The Fiddlesticks Affair" and "The Yo-Ho-Ho and a Bottle of Rum Affair." In 1962, he was cast as Glenn Kassin in "The Earth Mover" episode of the modern NBC western series, Empire.

In 1963–1964, he appeared as the wandering gold-seeker father, "Doc" Sardius McPheeters, in the ABC western series The Travels of Jaimie McPheeters with co-star Kurt Russell as Doc's son, Jaimie. On another ABC series, The Long, Hot Summer, O'Herlihy became the lead star, having replaced Edmond O'Brien in the part of Will Varner midway through the program's one-season run. In 1966, he appeared in the episode "Have You Seen the Aurora Borealis?" of NBC's western series The Road West, starring Barry Sullivan. In 1974 he appeared in QB VII, and played the Senior American Officer, Col. Max Dodd in the second series of BBC's POW drama Colditz. Hawaii Five O 1975. In 1976, he guest-starred in an episode of NBC's dramatic series Gibbsville. In 1978, he guest starred in the second part of the Battlestar Galactica episode "Gun on Ice Planet Zero" as Dr. Ravishol. O'Herlihy also portrayed the ill-fated lumber tycoon Andrew Packard in the cult television program Twin Peaks (1991), and in the Batman: The Animated Series episode "Deep Freeze", voicing the villainous theme park mogul Grant Walker. In 1998, O'Herlihy acted in his last film, The Rat Pack, playing Joseph P. Kennedy.

O'Herlihy died of natural causes in Malibu, California in 2005, aged 85. His personal papers are held in the University College Dublin Archives.