Abraham was born Murray Abraham on October 24, 1939 in Pittsburgh, Pennsylvania, the son of Fahrid "Fred" Abraham, an auto mechanic, and his wife Josephine (née Stello), a housewife. His father was Assyrian and emigrated from Syria at age 5 during the famine; his paternal grandfather was a chanter in the Syriac Orthodox Church. His mother, one of 14 children, was Italian American, and the daughter of an immigrant who worked in the coal mines of Western Pennsylvania. He had two brothers, Robert and Jack, who were killed in separate car accidents.

Abraham was raised in El Paso, Texas, near the Mexican border. He attended Vilas Grammar School, and graduated from El Paso High School in 1958. He was a gang member during his teenage years. He attended Texas Western College (later named University of Texas at El Paso), where he was given the best actor award by Alpha Psi Omega for his portrayal of the Indian Nocona in Comanche Eagle during the 1959–60 season. He attended the University of Texas at Austin, then studied acting under Uta Hagen in New York City. He began his acting career on the stage, debuting in a Los Angeles production of Ray Bradbury's The Wonderful Ice Cream Suit.

Abraham has been married to Kate Hannan since 1962; they have two children, Mick and Jamili, and one grandchild, Hannan.

Abraham made his screen debut as an usher in the George C. Scott comedy They Might Be Giants (1971). By the mid-1970s, Murray had steady employment as an actor, doing commercials and voice-overs. Abraham can be seen as one of the undercover police officers along with Al Pacino in Sidney Lumet's Serpico (1973), and in television roles including the bad guy in one fourth-season episode of Kojak ("The Godson"). He played a cabdriver in the theatrical version of The Prisoner of Second Avenue (1975), a mechanic in the theatrical version of The Sunshine Boys (1975), and a police officer in the film All the President's Men (1976).

Despite these small roles, Abraham continued to do commercials and voice-over work for income. But in 1978, he decided to give them up. Frustrated with the lack of substantial roles, Abraham said, "No one was taking my acting seriously. I figured if I didn't do it, then I'd have no right to the dreams I've always had." His wife, Kate Hannan, went to work as an assistant and Abraham became a "house husband". He described, "I cooked and cleaned and took care of the kids. It was very rough on my macho idea of life. But it was the best thing that ever happened to me." Abraham appeared as drug dealer Omar Suárez alongside Pacino again in the gangster film Scarface (1983). He gained visibility voicing a talking bunch of grapes in a series of television commercials for Fruit of the Loom underwear.

F. Murray Abraham also joined The Mirror Theater Ltd's Mirror Repertory Company in 1984. He joined MRC the week after winning his Oscar for Best Actor for his work in Amadeus because he wanted to work with MRC Artist-in-Resience Geraldine Page, and would star opposite her in MRC’s The Madwoman of Chaillot.

Abraham won the Academy Award for Best Actor for his role as envious Composer Antonio Salieri in Amadeus (1985), an award for which Tom Hulce, playing Mozart in that movie, had also been nominated. He won a Golden Globe Award, among other awards, and his role in the film, directed by Miloš Forman, is still his most iconic. (He later continued the classical-music theme by narrating the plot summaries of the operas of Wagner's Ring Cycle in the 1990 PBS broadcast from the Metropolitan Opera, to the largest viewing audience of the Ring Cycle in history, conducted by James Levine.)

After Amadeus, he next appeared in The Name of the Rose (1986), in which he played Bernardo Gui, nemesis to Sean Connery's william of Baskerville. In its DVD commentary, his Director on the film, Jean-Jacques Annaud, described Abraham as an "egomaniac" on the set, who considered himself more important than Sean Connery, since Connery did not have an Oscar. That said, the film was a critical success. Abraham had tired of appearing as heavies and wanted to return to his background in comedy, as he explained to People Weekly Magazine in an interview he gave at the time of its release.

In 1994, Abraham portrayed Roy Cohn in the first Broadway production of Tony Kushner's Angels in America at the Walter Kerr Theater, replacing Ron Leibman in the role.

Though Abraham had fewer prominent roles in the next decade or so, he became known for his roles in Woody Allen's Mighty Aphrodite (1995), Ahdar Ru'afo in Star Trek: Insurrection (1998), and Gus Van Sant's Finding Forrester (2000), where he once again played nemesis to Connery. He had a significant role in The Bonfire of the Vanities (1990), but chose not to be credited due to a contract dispute.

In July 2004, during a ceremony in Rome, he was awarded the "Premio per gli Italiani nel Mondo". This is a prize distributed by the Marzio Tremaglia foundation and the Italian government to Italian emigrants and their descendants who have distinguished themselves abroad.

Abraham has focused on stage work throughout his career, giving notable performances as Pozzo in Mike Nichols's production of Waiting for Godot, Malvolio in Twelfth Night for the New York Shakespeare Festival, and Shylock in The Merchant of Venice for the Off-Broadway Theatre for a New Audience (TFANA) in March 2007, which was performed at the Duke Theatre in New York and also at the Swan Theatre, part of the Royal Shakespeare Company. He reprised this role in February 2011, when he replaced Al Pacino in the Public Theater's production. In the 1997/98 Broadway season, he starred in the new chamber musical Triumph of Love opposite Betty Buckley, based on Marivaux's classic comedy. The production did not find a large audience, running 85 performances, after its pre-opening preview period. He has also taught theater at Brooklyn College. In 2016, he played the title role in Classic Stage Company's production of Nathan the Wise.

Abraham has spoken about his faith: "I've attended many churches. I grew up as an Orthodox Christian and I was an altar boy. I love the Society of Friends, the Quakers. I attended their meetings for almost 15 years. I'm now [in 2008] attending the First Presbyterian Church of New York because they're such a generous, terrific church with outreach. They reach out to old people, to homeless, to A.A., to cross-dressers; it's truly a church of the teachings of Christ. Religion is essential to my life."

In 2009, he was recognized by the Alumni Association of the City College of New York with John H. Finley Award in recognition of exemplary dedicated Service to the City of New York.

In 2010, Abraham was the recipient of The Gielgud Award (Theatre) for that year.

In 2015, Abraham was an inductee to the American Theater Hall of Fame.