Dostum was born in 1954 in Khwaja Du Koh near Sheberghan in Jowzjan Province, Afghanistan. Coming from an impoverished Uzbek family, he received a very basic traditional education as he was forced to drop out of school at a young age. From there, he took up work in the village's major gas fields.

Dostum began working in 1970 in a state-owned gas refinery in Sheberghan, participating in union politics, as the new government started to arm the staff of the workers in the oil and gas refineries. The reason for this was to create "groups for the defense of the revolution". Because of the new communist ideas entering Afghanistan in the 1970s, he enlisted in the Afghan National Army in 1978. Dostum received his basic military training in Jalalabad. His squadron was deployed in the rural areas around Sheberghan, under the auspices of the Ministry of National Security.

During the Soviet-Afghan War, Dostum was commanding a militia battalion to fight and rout mujahideen forces; he had been appointed an officer due to prior military experience. This eventually became a regiment and later became incorporated into the defense forces as the 53rd Infantry Division. Dostum and his new division reported directly to President Mohammad Najibullah. Later on he became the commander of the military unit 374 in Jowzjan. He defended the Soviet-backed Afghan government against the mujahideen forces throughout the 1980s. While he was only a regional commander, he had largely raised his forces by himself. The Jowzjani militia Dostum controlled was one of the few in the country which was able to be deployed outside its own region. They were deployed in Kandahar in 1988 when Soviet forces were withdrawing from Afghanistan.

Dostum's men would become an important force in the fall of Kabul in 1992. In April 1992, the opposition forces began their march to Kabul against the government of Najibullah. Dostum had allied himself with the opposition commanders Ahmad Shah Massoud and Sayed Jafar Naderi, the head of the Isma'ili community, and together they captured the capital city. He and Massoud fought in a coalition against Gulbuddin Hekmatyar. Massoud and Dostum's forces joined together to defend Kabul against Hekmatyar. Some 4000-5000 of his troops, units of his Sheberghan-based 53rd Division and Balkh-based Guards Division, garrisoning Bala Hissar fort, Maranjan Hill, and Khwaja Rawash Airport, where they stopped Najibullah from entering to flee.

Dostum then left Kabul for his northern stronghold Mazar-i-Sharif, where he ruled, in effect, an independent region (or 'proto-state'), often referred as the Northern Autonomous Zone. He printed his own Afghan currency, ran a small airline named Balkh Air, and formed relations with countries like Uzbekistan. While the rest of the country was in chaos, his region remained prosperous and functional, and it won him the support from people of all ethnic groups. Many people fled to his territory to escape the violence and fundamentalism imposed by the Taliban later on. In 1994, Dostum allied himself with Gulbuddin Hekmatyar against the government of Burhanuddin Rabbani and Ahmad Shah Massoud, but in 1995 sided with the government again.

Following the rise of the Taliban and their capture of Kabul, Dostum aligned himself with the Northern Alliance (United Front) against the Taliban. The Northern Alliance was assembled in late 1996 by Dostum, Massoud and Karim Khalili against the Taliban. At this point he is said to have had a force of some 50,000 men supported by both aircraft and tanks.

Much like other Northern Alliance Leaders, Dostum also faced infighting within his group and was later forced to surrender his power to General Abdul Malik Pahlawan. Malik entered into secret negotiations with the Taliban, who promised to respect his authority over much of northern Afghanistan, in exchange for the apprehension of Ismail Khan, one of their enemies. Accordingly, on 25 May 1997 Malik arrested Khan, handed him over and let the Taliban enter Mazar-e-Sharif, giving them control over most of northern Afghanistan. Because of this, Dostum was forced to flee to Turkey. However, Malik soon realized that the Taliban were not sincere with their promises as he saw his men being disarmed. He then rejoined the Northern Alliance, and turned against his erstwhile allies, driving them from Mazar-e-Sharif. In October 1997, Dostum returned from exile and retook charge. After Dostum briefly regained control of Mazar-e-Sharif, the Taliban returned in 1998 and he again fled to Turkey.

Dostum is considered to be liberal and somewhat leftist. Being ethnic Uzbek, he has worked on the battlefield with Leaders from all other major ethnic groups, Hazaras, Pashtuns and Tajiks. When Dostum was ruling his northern Afghanistan proto-state before the Taliban took over in 1998, women were able to go about unveiled, girls were allowed to go to school and study at the University of Balkh, cinemas showed Indian films, music played on television, and Russian vodka and German beer were openly available – activities which were all banned by the Taliban.

General Dostum has been accused by Western journalists of responsibility for the suffocating or otherwise killing of 2,000 Taliban prisoners in December 2001. Dostum denied the accusations in 2009. US President Obama in 2009 ordered an investigation into the matter, which as yet has yielded no (published) results.

Dostum served as deputy defense minister the early period of the Karzai administration. In March 2003, he established a North Zone of Afghanistan. On 20 May 2003, Dostum narrowly escaped an assassination attempt. He was often residing outside Afghanistan, mainly in Turkey. In February 2008 he was suspended after the apparent kidnapping and torture of a political rival.

He viewed the ISAF forces attempt to crush the Taliban as ineffective and has gone on record saying in 2007 that he could mop up the Taliban "in six months" if allowed to raise a 10,000 strong army of Afghan veterans. Senior Afghan government officials do not trust Dostum as they are concerned that he might be secretly rearming his forces.

Some media reports in 2008 stated earlier that Dostum was "seeking political asylum" in Turkey while others said he was exiled. One Turkish media outlet said Dostum was visiting after flying there with then Turkey's Foreign Minister Ali Babacan during a meeting of the Organization for Security and Cooperation in Europe (OSCE).

On 16 August 2009, Dostum made a requested return from exile to Afghanistan to support President Hamid Karzai in his bid for re-election. He later flew by helicopter to his northern stronghold of Sheberghan, where he was greeted by thousands of his supporters in the local stadium. He subsequently made overtures to the United States, promising he could "destroy the Taliban and al Qaeda" if supported by the U.S., saying that "the U.S. needs strong friends like Dostum."

After the fall of the Taliban he joined Hamid Karzai's presidential administration but spent most of his time in Turkey. He also served as Chairman Joint Chiefs of Staff of the Afghan Army, a role often viewed as ceremonial. From 2011 he was part of the leadership council of the National Front of Afghanistan along with Ahmad Zia Massoud and Mohammad Mohaqiq. In 2014 he joined Ashraf Ghani's presidential administration as a vice President, but was forced to flee again in 2017 after being accused of sexually assaulting a political rival.

Dostum was directly chosen as First Vice President of Afghanistan in the April–June 2014 Afghan presidential election, next to Ashraf Ghani as President and Sarwar Danish as second Vice President.

In July 2016 Human Rights Watch accused Abdul Rashid Dostum's National Islamic Movement of Afghanistan of killing, abusing and looting civilians in the northern Faryab Province during June. Militia forces loyal to Dostum stated that the civilians they targeted - at least 13 killed and 32 wounded - were supporters of the Taliban. In 2017 he was accused of kidnapping and sexually assaulting a political rival, claims that he denies and that forced him into exile in Turkey.