Bashir was born in Jombang, East Java, on 17 August 1938, to a family of Hadhrami Arab and Javanese descent. He was a student of Gontor Islamic boarding school in Ponorogo, graduating in 1959, before entering Al-Irsyad University, in Solo, Central Java, and graduating in 1963. After time as an Activist for the Islamic Student Association (Indonesian: Himpunan Mahasiswa Islam) in Solo, he was elected secretary of Al-Irsyad Youth Organization, and then President of Indonesian Islamic Youth Movement (GPII) (1961), and Indonesian Student Da'wah Organization (LDMI).

In 1972, Bashir founded Al-Mukmin boarding school with friends Abdullah Sungkar, Yoyo Roswadi, Abdul Qohar H. Daeng Matase and Abdllah Baraja. Al-Mukmin is located in Ngruki, near Solo, Central Java. Initially, Al-Mukmin's activities were limited to religious discussion after dhuhr (mid-day prayer). Following increasing interest, the founders expanded Al-Mukmin into Madrasah (Islamic school) and then to Pesantren (Islamic boarding school).

Bashir returned to Indonesia in 1999 and became a cleric, renewing his call for Sharia law. Ba'asyir has two sons—Abdul Roshid Ridyo Ba'asyir, born 31 January 1974 in Sukarjo, Java, Indonesia; and Abdul Rahim Ba'asyir, born 16 November 1977 in Surakarta, Java, Indonesia— and a daughter, Zulfur.

During Indonesian President Suharto's New Order, Bashir and Sungkar were arrested for a number of reasons, firstly for actively supporting Sharia, the non-recognition of the Indonesian national ideology Pancasila which in part promotes religious pluralism. Secondly, the refusal of their school to salute the Indonesian flag which signified Bashir's continual refusal to recognise the authority of a secular Indonesian state. Bashir appealed but was subsequently imprisoned without trial from 1978 to 1982. Soon after his release, Bashir was convicted on similar charges; he was also linked to the bomb attack on the Buddhist monument Borobudur in 1985 but fled to Malaysia. During his years in exile Bashir undertook religious teachings in both Malaysia and Singapore. The United States government alleged that during this period he became involved with Jamaah Islamiyah, an alleged militant Islamist group. Bashir remained in exile until Indonesian President Suharto's fall in 1998.

Similarly, US State Department translator Fred Burks revealed during Bashir's trial in Indonesia that the USA government had asked President Megawati secretly to hand-over Bashir in a meeting at Megawati's home in September 2002. Present at that meeting was US ambassador to Indonesia Ralph L. Boyce, National Security Council official Karen Brooks, and an unnamed CIA official.

On 14 April 2003, he was formally charged by the Indonesian government with treason, immigration violations, and providing false documents and statements to the Indonesian police. The charges are mainly related to the 2000 Christmas Eve bombings against Christian churches, which killed 18 people. In the Indonesian court, he was found not guilty of treason because the state failed to prove its case, but was found guilty on the immigration violations. In a local TV news interview, Metro TV, when asked, 'Are you truly a terrorist?'; He simply answered, 'No, I've never killed anyone.' He was sentenced to three years in prison, but the sentence was subsequently reduced to 20 months due to his good behavior in the prison.

On 15 October 2004, he was arrested by the Indonesian authorities and charged with involvement in the bomb attack on the Marriott Hotel in Jakarta on 5 August 2003, which killed 14 people. Secondary charges in the same indictment charge him of involvement in the 2002 Bali bombing, the first time he has faced charges in relation to that attack which killed 202 people. On March 3, 2005, Bashir was found guilty of conspiracy over the 2002 attacks, but was found not guilty of the charges surrounding the 2003 bombing. He was sentenced to two and a half years imprisonment.

On 17 August 2005, as part of the tradition of remissions for Indonesia's Independence Day, Bashir's jail term was cut by 4 months and 15 days. On 14 June 2006, to cheers from his supporters waiting outside, Abu Bakar Bashir was released, having served 25+ months in Jakarta's Cipinang prison, where he held court and coordinated the publication of a commemorative book with his release. About forty Bodyguards in uniform black jackets linked arms to escort Bashir through chanting crowds.

On December 21, 2006, Bashir's conviction was overturned by Indonesia's Supreme Court. He publicly criticised the United Nations because he remained on the body's list of international terrorists, saying "I am terrorist number 35 on the list."

In October 2008, Bashir announced he intends to start a new Islamic group in Indonesia, JAT or Jamaah Ansharut Tauhid ("partisans of the oneness of God"), at the time the Indonesian government was preparing to execute the three Bali bombers. Bashir repeated his claim that a nuclear device was released by the CIA, saying the attack was conspiracy between "America, Australia and the Jews" In February 2012, the US Department of State website stated that JAT was responsible for multiple coordinated attacks against innocent civilians, police and military personnel in Indonesia. "JAT has robbed banks and carried out other illicit activities to fund the purchase of assault weapons, pistols and bomb-making materials", so JAT is put on US terror list.

On 13 December 2010, Indonesian police charged Bashir with involvement in plans of terror and military training in Aceh. The charge against him of inciting others to commit terrorism, carries the death penalty.

On 16 June 2011, Bashir was convicted of supporting a jihadi training camp following a four-month trial. He was sentenced to 15 years in prison. He was acquitted of the charge of involvement in the 2002 Bali bombings. However, after an appeal the Jakarta High Court reduced his sentence to nine years. Finally, the Supreme Court has rejected the appeal of Abu Bakar Bashir and annulled the Jakarta High Court sentence then reinstated the South Jakarta District Court's original 15-year sentence.

In July 2014, an imprisoned Bashir pledged allegiance, or Bay'ah, to the militant Islamic State of Iraq and the Levant (ISIL) and its leader, Abu Bakr al-Baghdadi.