After the failure of the Shichigatsu-za, Shimura went back to Osaka, where he began to get roles in radio plays. In 1930 he joined the Kindaiza (近代座) theatre company and became a fully professional actor. He toured China and Japan with the Kindaiza, but in 1932 he left the company and returned again to Osaka, where he appeared with the Shinseigeki (新声劇) and Shinsenza (新選座) troupes.　Talking pictures were just then coming in and Shimura realised they would provide opportunities for stage-trained actors. In 1932 he joined the Kyoto studios of the film production company Shinkō Kinema. He made his film debut in the 1934 silent Ren'ai-gai itchōme (恋愛街一丁目: Number One, Love Street). The first film in which he had a speaking part was the 1935 Chūji uridasu (忠次売出す), directed by Mansaku Itami. His first substantial film role was as a detective in Mizoguchi Kenji's 1936 Osaka Elegy (Naniwa erejii; 浪華悲歌).

The film which established his reputation as a first-rate actor was Itami Mansaku's 1936 Akanishi Kakita (赤西蠣太: Capricious Young Man). In 1937 he moved to Nikkatsu film corporation's Kyoto studios, and between then and 1942 appeared in nearly 100 films. His most notable role in these years was that of Keishirō in the long-running series Umon Torimono-chō (右門捕物帖), starring Kanjūrō Arashi. He also demonstrated his considerable ability as a singer in the 1939 "cine-operetta", Singing Lovebirds. During this time the political regime in Japan was growing ever more oppressive, and Shimura was arrested by the Special Higher Police (Tokubetsu Kōtō Keisatsu, known as Tokkō) and held for about three weeks because of his earlier association with left-wing theatre groups. He was eventually released on the recognisance of his wife Masako and fellow-actor Ryūnosuke Tsukigata.　He is said to have made use of this experience later when playing a Tokkō official in Akira Kurosawa's 1946 No Regrets for Our Youth. When Nikkatsu and Daiei merged in 1942, Shimura moved to the Kōa Eiga studios and then in 1943 to Tōhō. A few weeks before the end of the Pacific War in August 1945, Shimura's elder brother was killed in Southeast Asia.

In 1943, Shimura appeared as the old jujutsu Teacher Murai Hansuke in Kurosawa's debut movie, Sanshiro Sugata. Along with Toshirō Mifune, Shimura is the actor most closely associated with Kurosawa: he eventually appeared in 21 of Kurosawa's 30 films. In fact, Kurosawa's cinematic collaboration with Shimura, from Sanshiro Sugata in 1943 to Kagemusha in 1980, started earlier and lasted longer than his work with Mifune (1948–65). Shimura's performances for Kurosawa included the Doctor in Drunken Angel (1948), the veteran detective in Stray Dog (1949), the flawed Lawyer in Scandal (1950), the woodcutter in Rashomon (1950), the mortally ill bureaucrat in Ikiru (1952), and the lead samurai Kambei in Seven Samurai (1954). Kurosawa wrote the part in Kagemusha specifically for Shimura, but the scenes were cut from the Western release, so many in the West did not know that he had been in the film. The DVD release of the film by The Criterion Collection restored Shimura's footage.

Shimura appeared in a number of Tōhō kaiju (giant monster) and tokusatsu (special effects) films, many of which were directed by Kurosawa's good friend and colleague Ishiro Honda. Shimura's roles included Professor Kyohei Yamane in the original Godzilla (1954), a character he briefly reprised in Godzilla Raids Again (1955).

Shimura died on February 11, 1982 in Tokyo, Japan, from emphysema at the age of 76. His effects were presented to the Film Centre of the National Museum of Modern Art, Tokyo.