In 1914, Veidt met Actress Lucie Mannheim, with whom he began a relationship. Later in the year Veidt was conscripted into the German Army during World War I. In 1915, he was sent to the Eastern Front as a non-commissioned officer and took part in the Battle of Warsaw. He contracted jaundice and pneumonia, and had to be evacuated to a hospital on the Baltic Sea. While recuperating, he received a letter from Mannheim telling him that she had found work at a theatre in Libau. Intrigued, Veidt applied for the theatre as well. As his condition had not improved, the army allowed him to join the theatre so that he could entertain the troops. While performing at the theatre, he ended his relationship with Mannheim. In late 1916, he was re-examined by the Army and deemed unfit for service; he was given a full discharge in January 1917. Veidt returned to Berlin to pursue his acting career.

From 1916 until his death, Veidt appeared in more than 100 films. One of his earliest performances was as the murderous somnambulist Cesare in Director Robert Wiene's The Cabinet of Dr. Caligari (1920), a classic of German Expressionist cinema, with Werner Krauss and Lil Dagover. His starring role in The Man Who Laughs (1928), as a disfigured circus performer whose face is cut into a permanent grin, provided the (visual) inspiration for the Batman villain the Joker, created in 1940 by Bill Finger. Veidt also starred in other silent horror films such as The Hands of Orlac (1924), another film directed by Robert Wiene, The Student of Prague (1926) and Waxworks (1924) where he played Ivan the Terrible.

Conrad Veidt married three times: he first married Augusta Holl, a cabaret entertainer known as "Gussy", on 18 June 1918. They divorced four years later. Gussy later married German actor Emil Jannings. Veidt's second wife Felicitas Radke was from an aristocratic German family; they married in 1923. Their daughter, Vera Viola Maria, called Viola, was born on 10 August 1925. He last married Ilona Prager, a Hungarian Jew called Lily, in 1933; they were together until his death.

Veidt also appeared in Magnus Hirschfeld's film Anders als die Andern (Different from the Others, 1919), one of the first films to sympathetically portray homosexuality, although the characters in it do not end up happily. He had a leading role in Germany's first talking picture, Das Land ohne Frauen (Land Without Women, 1929).

He moved to Hollywood in the late 1920s and made a few films, but the advent of talking pictures and his difficulty with speaking English led him to return to Germany. During this period he lent his expertise to tutoring aspiring performers, one of whom was the later American character Actress Lisa Golm.

Goebbels had imposed a "racial questionnaire" in which everyone employed in the German film industry had to declare their "race" to continue to work. When Veidt was filling in the questionnaire, he answered the question about what his Rasse (race) was by writing that he was a Jude (Jew). Veidt was not Jewish, but his wife was Jewish, and Veidt would not renounce the woman he loved. Additionally, Veidt who was opposed to anti-Semitism wanted to show solidarity with the German Jewish community, who were rapidly being stripped of their rights as German citizens in the spring of 1933. As one of Germany's most popular actors, Veidt had already been informed that if he was prepared to divorce his wife and declare his support for the new regime, he could continue to act in Germany. Several other leading actors who had been opposed to the Nazis before 1933 switched allegiances. In answering the questionnaire by stating he was a Jew, Veidt rendered himself unemployable in Germany, but stated this sacrifice was worth it as there was nothing in the world that would compel him to break with his wife. Upon hearing about what Veidt had done, Goebbels remarked that he would never act in Germany again.

After arriving in Britain, he perfected his English and starred in the title role of the original anti-Nazi version of Lion Feuchtwanger's novel, Jew Süss (1934) directed by German-born US Director Lothar Mendes and produced by Alexander Korda for Michael Balcon's Denham Studio. He became a British citizen by 1938. By this point multi-lingual, Veidt made films in both French with expatriate French Directors and in English, including three of his best-known roles for British Director Michael Powell in The Spy in Black (1939), Contraband (1940) and The Thief of Bagdad (1940).

He starred in a few films, such as George Cukor's A Woman's Face (1941) where he received billing just under Joan Crawford's and Nazi Agent (1942), in which he had a dual role as both an aristocratic German Nazi spy and as the man's twin brother, an anti-Nazi American. His best-known Hollywood role was as the sinister Major Heinrich Strasser in Casablanca (1942), a film which was written and began pre-production before the United States entered World War II.

Conrad Veidt died on 3 April 1943 of a massive heart attack while playing golf at the Riviera Country Club in Los Angeles with singer Arthur Fields and his personal physician, Dr. Bergman, who pronounced him dead on the scene. Veidt was 50 years old. In 1998, his ashes were placed in a niche of the columbarium at the Golders Green Crematorium in north London.