La Rue went from high school to his first acting job, in Otis Skinner's road company production of Blood and Sand. He performed in Broadway plays from around 1923 to 1931. According to La Rue, while appearing in Mae West's play Diamond Lil, he was spotted by Howard Hawks, who offered him a part in the film Scarface (1932), starring Paul Muni.

He moved to Hollywood, where he appeared in numerous films. However, Scarface was not one of them. La Rue stated in a newspaper article that, after four days, Hawks had to replace him with George Raft because La Rue was taller than Muni and had a more powerful voice. Later, however, Raft turned down the role of the despicable villain in The Story of Temple Drake (1933), fearing it would damage his screen image, so the part went to La Rue. Sometimes mistaken for Humphrey Bogart, he played thugs and Gangsters for the most part. However, Director Frank Borzage atypically cast him as a priest in the 1932 version of A Farewell to Arms simply because, according to newspaper columnist Hubbard Keavy, he was "tired of seeing conventional characters". La Rue stated he turned down a role in The Godfather (1972) and many parts in the television series The Untouchables because of the way they portrayed Italian-Americans.

He was married three times. La Rue married Los Angeles socialite Constance Deighton Simpson on September 22, 1938, in London. She obtained a divorce on December 17, 1946, charging him with mental cruelty. In 1955, he obtained an annulment from former Baroness Violet Edith von Rosenberg after six years of marriage, claiming she had only married him to obtain American citizenship and that they separated after less than two months. He married Anne Giordano on August 12, 1962; she obtained an annulment in 1967. Jack La Rue had no children.