He took part in the National Youth Theatre summer school in 1964, but his background and his northern accent made him feel out of place. Armstrong auditioned for RADA but was not accepted. He instead studied fine art at Newcastle University. He found the course pretentious and felt that he did not fit in, and he was expelled after two years because he stopped attending classes.

Armstrong has played over 80 different roles in television productions in the course of his career. During the 1970s, he appeared in various TV series, including episodes of Whatever Happened to the Likely Lads?, Porridge and The Sweeney.

In addition to his film and television work, Armstrong has acted in many theatre productions. One of his early roles was Billy Spencer in David Storey's play The Changing Room at the Royal Court Theatre directed by Lindsay Anderson in 1971. In 1975, he played Touchstone in As You Like It directed by Peter Gill at the Nottingham Playhouse.

He was cast in two mini-series dealing with coal miners in North East England. He played Joe Gowlan in The Stars Look Down (1974) based on the novel by A. J. Cronin, and he appeared in Ken Loach's Days of Hope (1975) set in his native County Durham. In a 2007 interview, Armstrong singled out Days of Hope as a favourite: "I loved that because it was my own history and background that was being dramatised and, in a way, nothing gets better than that."

In the comedy series A Sharp Intake of Breath, he played a variety of characters who complicate the life of the main character played by David Jason. In 1977, he was the strict Deputy Headmaster in Willy Russell's Our Day Out, a television play about a group of underprivileged students on a daytrip. He also starred in the 1981 Yorkshire Television drama Get Lost!

Armstrong spent nine years with the Royal Shakespeare Company from 1979 to 1988. On tour and at the Donmar Warehouse in 1979–80, he played Dogberry in Much Ado About Nothing and Azdak in The Caucasian Chalk Circle.

In 1981, Armstrong joined the cast of the eight-hour production of The Life and Adventures of Nicholas Nickleby as Wackford Squeers. The company went on tour to perform on Broadway at the Plymouth Theatre. The play was filmed for television at the Old Vic Theatre in 1982.

In productions at the Royal Shakespeare Theatre, the Theatre Royal, Newcastle, and the Barbican Theatre in 1982–83, Armstrong played Trinculo in The Tempest and Petruchio in The Taming of the Shrew with Sinéad Cusack as Kate. In 1983, he played Ralph Trapdoor in The Roaring Girl starring Helen Mirren. He performed the roles of Leontes in The Winter's Tale and John Proctor in The Crucible on a national tour that included Christ Church, Spitalfields in 1984 and on tour to Poland in 1985. In 1985–86, he played Thersites in Troilus and Cressida.

Armstrong received nominations in two categories for the 1985 Olivier Award: Outstanding Performance by an Actor in a Musical for Les Misérables and Actor of the Year for The Crucible and The Winter's Tale. In 1988, he was again nominated for the Olivier Award for the roles of Barabas in an RSC production of The Jew of Malta and the Captain in a National Theatre production of The Father by August Strindberg. The New York Times review of The Father said: "At its imploding center is the superb actor Alun Armstrong... 'To eat or be eaten, that is the question,' says the captain. By evening's end, Mr. Armstrong seems to have been devoured alive by his inner demons..."

His first cinematic lead role was as Maxwell Randall, the titular green baize vampire in the Alan Clarke directed snooker musical Billy the Kid and the Green Baize Vampire (1987). Armstrong sang "I bite back".

During the short run of the musical The Baker's Wife at the Phoenix Theatre in 1989–90, he played the role of the baker Aimable Castagnet. The production, directed by Trevor Nunn, received positive reviews but did not attract large audiences and closed after 56 performances. He was nominated for an Olivier Award for Outstanding Performance of the Year by an Actor in a Musical.

In Patriot Games (1992), Armstrong played an SO-13 officer. In Braveheart (1995), he played the Scottish noble Mornay who betrayed william Wallace. He was the villainous Egyptian cult leader Baltus Hafez in The Mummy Returns (2001), and he portrayed Saint Peter with a Geordie accent in Millions (2004). He also had small roles as the High Constable in Sleepy Hollow (1999), Cardinal Jinette in Van Helsing (2004), Magistrate Fang in Roman Polanski's Oliver Twist (2005) and Uncle Garrow in Eragon (2006).

Armstrong won the Olivier Award for Best Actor in a Musical in 1994 for his performance as Sweeney Todd in the 1993 London revival of the musical at the National Theatre. The play also won for Best Musical Revival and his co-star Julia McKenzie won Best Actress in a Musical.

At the Donmar Warehouse, Armstrong appeared as Albert Einstein in Terry Johnson's Insignificance in 1995, and he played Hamm in Samuel Beckett's Endgame in 1996. He starred as Willy Loman in a 1996–97 National Theatre production of Death of a Salesman. In 1997–98, he appeared in a production of the comedy The Front Page directed by Sam Mendes at the Donmar Warehouse. The Independent review noted: "As for Alun Armstrong, we don't meet him until late in the second of three acts but he dominates the entire evening. He barks, bleats and bellows across the stage, grabbing Hildy and the show by the scruff of the neck and hurtling through to a zinger of a climax."

In the BBC drama series Our Friends in the North (1996), he played Austin Donohue, a character based on the Politician T. Dan Smith. Armstrong portrayed 18th century Politician Henry Fox in the BBC serial Aristocrats (1999). In the 2000 TV film This Is Personal: The Hunt for the Yorkshire Ripper, he portrayed George Oldfield, the Assistant Chief Constable for Crime at West Yorkshire Police whose health deteriorated during the investigation as he received messages purportedly from the killer. He was nominated for a Royal Television Society award for his role in This Is Personal.

Armstrong took the lead role at short notice in Shelagh Stephenson's play Mappa Mundi in 2002, replacing Ian Holm who withdrew due to illness. In 2006, he returned to the stage to star in Trevor Nunn's production of The Royal Hunt of the Sun at the National Theatre. At the Proms in 2012, he played Alfred Doolittle in a performance of My Fair Lady starring Annalene Beechey and Anthony Andrews. Armstrong stars in a 2014 production of Ionesco's black comedy Exit the King at the Theatre Royal, Bath's Ustinov Studio.

During the run of New Tricks, Armstrong continued to take on other projects. He starred in the 2004 TV film When I'm 64 about a lonely retired schoolteacher who starts a relationship with another man. He chose the role, despite his apprehension about filming a love scene with co-star Paul Freeman, because he thought it was a lovely and thought-provoking story. He also starred in The Girls Who Came to Stay (2006), about a British couple who take in two girls exposed to the effects of the Chernobyl disaster, and Filth (2008), as the husband of "Clean-Up TV" Activist Mary Whitehouse.

In July 2009, Armstrong was awarded two honorary degrees in recognition of his contributions to the arts. He received an Honorary Doctorate of Letters from the University of East Anglia and an Honorary Doctorate of Arts from the University of Sunderland. The theatre at the Civic Hall in Stanley, County Durham, near Armstrong's hometown, was named after him in 2014.

Armstrong and his wife, Sue, have three sons: Tom, Joe, and Dan. Joe is also an actor. Father and son played older and younger versions of the same character in the 2010 BBC drama A Passionate Woman, and they played Northumberland and his son Hotspur in the 2012 BBC adaptation of Henry IV. Dan is a musician in the band Clock Opera. Armstrong appeared in the music video for their song "The Lost Buoys".

Armstrong is known for his role as Brian Lane in the BBC One series New Tricks about a group of former police detectives who help investigate unsolved and open cases for London's Metropolitan Police. The character of Brian Lane is an obsessive and socially inept recovering alcoholic who has a great capacity for remembering details of old cases and colleagues. In August 2012, Armstrong announced he would leave the show after the tenth series. The announcement followed comments by the cast in an interview with the Radio Times that criticised some of the series' writing, and which drew an angry rebuttal from the show's writer-director Julian Simpson.

Born Alan Armstrong in Annfield Plain, County Durham, his father was a coal miner and both his parents were Methodist lay Preachers. He attended Annfield Plain Junior School and then went on to Consett Grammar School, where a Teacher inspired him to try acting. In the lower sixth, he played Petruchio in The Taming of the Shrew – a role he would later perform with the Royal Shakespeare Company.