Jay was born in London to Jack and Ethel Jay (née Bart, born in Hendon in 1907), who married in 1930. When they married, the family name was Jacobs, but by the time Tony's brother Robert was born in 1945, the family name had changed to Jay. His family were Jewish.

Tony Jay attended Pinner County Grammar School, and completed his National Service with the Royal Air Force in 1953. He later recalled, "I was always an actor at school"; but opted for the financial security of a real estate Business. Jay and his first wife moved to South Africa in about 1966, after hearing of the potential there for his line of work. He left South Africa to return to England in 1973, after which he moved to the United States in 1986.

In various animated projects Jay took over the voice of Shere Khan, which actor George Sanders had originated for the 1967 Disney animated film The Jungle Book. In 11 episodes spanning 1990–91, Jay voiced Shere Khan for Disney's animated TV series TaleSpin, The Jungle Book: Rhythm and Groove videogame (2000), and the House of Mouse (2001–02). His final appearance as Khan came in the 2003 film The Jungle Book 2. His final role was voicing Spiderus in the Miss Spider series.

Jay acted, wrote, and directed radio plays on the South African Broadcasting Corporation's first commercial radio station, Springbok Radio (1950–85). He was especially associated with the comic series Taxi! (1969-1972,1975-1978), in which he not only portrayed New York cabby Red Kowalski, but also co-wrote many scripts with Joe Stewardson. Other shows in which he was involved included Lux Radio Theatre, Playhouse 90, and Tuesday Theatre. Jay adapted, cast, and directed the first 6 months of episodes for The Avengers. The series, based on Seasons 4-6 of the 1960s British television series of the same name, was broadcast on Springbok from 6 December 1971 to 28 December 1973. To bridge the gap between the visual orientation of the British television series and the sound-only perspective of radio, Jay created a narrator which he embued with irony and scepticism.

After Jay's return to England in 1973 he worked in various television productions. For the BBC series Fall of Eagles (1974) he portrayed Tsar Alexander III of Russia, during which time he met Patrick Stewart, who played Vladimir Lenin. Jay appeared as the merchant in Shakespeare's Timon of Athens (1981) for BBC Shakespeare, and in single episodes of popular television programmes such as The Sweeney (1975), The Professionals (1978) and the comedy Whoops Apocalypse (1982).

During this period he was cast as Vladimir Maximovitch in Woody Allen's Love and Death (1975), which was shot in Hungary and France. George Lucas met with Jay about playing Obi-Wan Kenobi in a planned film with the working title Galactic Warfare. Despite Jay agreeing to the role, Lucas decided to cast Alec Guinness, instead.

On stage he had a small roles in plays such as Anton Chekhov's The Three Sisters (1976) and Terence Rattigan's The Deep Blue Sea (1981). More important roles followed with an appearance as Jaggers in Great Expectations (1984) at the Old Vic, and as Shylock in The Merchant of Venice.

During its run from August to October 1986 at the Broadhurst Theatre on Broadway, Jay's performance was described as "brilliantly played" by the New York Times. Consequently, he was nominated for the 1987 Drama Desk Award as Outstanding Featured Actor in a Play.

He wed Married With Children make-up Artist Kathy Rogers in 1988, and they had a son, Adam, in December that year. In 2004 he married Marta MacGeraghty.

Jay's voice-over work included Monsieur D'Arque, the amoral asylum superintendent, in Disney's 1991 hit animated film version of Beauty and the Beast. From 1994–2001 he supplied the voice for the virus Megabyte in the computer animated television show ReBoot. According to one source, Jay was preferred to Patrick Stewart, Derek Jacobi, Ian McKellen for the voice of Judge Claude Frollo in Disney's 1996 animated film adaptation of The Hunchback of Notre Dame, directed by Gary Trousdale and Kirk Wise, who had also steered his performance in the film version of Beauty and the Beast. Jay reprised Frollo's voice for Walt Disney World's nighttime light and fireworks show Fantasmic! From 1995 to 1996 Jay was the voice of the alien warlord Lord Dregg, the villain during the last two seasons of the original 1987 Teenage Mutant Ninja Turtles animated TV series.

Jay also made many distinguished on-screen appearances, including the role of Nigel St John on Lois and Clark: The New Adventures of Superman (1993–1995). He further made guest appearances on programs including The Golden Girls in 1987, Twin Peaks in 1990-91, and Night Court in 1991. His most prominent onscreen role, however, is that of Professor Werner in Twins (1988), the mentor to Arnold Schwarzenegger's protagonist, Julius Benedict.

He is also well-known among fans of the 1996–2003 video game series Legacy of Kain for his voicing of the original Mortanius and of the Elder God, alongside several other minor characters.

Jay was a devotee of classic Broadway and made several recordings and performances of old-time Broadway lyrics, in spoken-word form. A CD of these readings, Speaking of Broadway, was released in 2005; a version recorded in 1996 was entitled Poets on Broadway, as was his website. On it Jay recites lyrics written by the likes of Noël Coward, Ira Gershwin, and Oscar Hammerstein, accompanied by synthesized music which he composed.

In April 2006 Jay underwent surgery in Cedars-Sinai Medical Center (Los Angeles) to remove a non-cancerous tumor from his lungs. Afterwards he became critically ill and was readmitted to Cedar Sinai, where he died on August 13, 2006, aged 73. He is buried in Forest Lawn Memorial Park in Los Angeles, California.