Mueller was born at Rostock, in the Grand Duchy of Mecklenburg-Schwerin. After the early death of his parents, Frederick and Louisa, his grandparents gave him a good education in Tönning, Schleswig. Apprenticed to a Chemist at the age of 15, he passed his pharmaceutical examinations and studied botany under Professor Ernst Ferdinand Nolte (1791–1875) at Kiel University. In 1847, he received his degree of Doctor of Philosophy from Kiel for a thesis on the plants of the southern regions of Schleswig.

He arrived at Adelaide on 18 December 1847 and found employment as a Chemist with Moritz J. Heuzenroeder, in Rundle Street. He was an inveterate Explorer, walking alone to Mount Arden and Mount Brown during his first year. Shortly afterwards, he obtained 20 acres (81,000 m) of land not far from Adelaide in the Bugle Ranges, and had a cottage built there. He moved there with his sister Clara, intending to start a farm, but after a few months, he returned to his former employment.

Mueller thought to open a chemist's shop in the gold diggings, so in 1851, he moved to Melbourne, capital of the new colony of Victoria. He had contributed a few papers on botanical subjects to German periodicals, and in 1852, sent a paper to the Linnean Society of London on "The Flora of South Australia", thus beginning to be well known in botanical circles.

Mueller was appointed government Botanist for Victoria by Governor Charles La Trobe in 1853, a post that was newly created for him. He examined its flora, especially the Alpine vegetation of Australia, which was previously unknown. He explored the Buffalo Ranges, then went to the upper reaches of the Goulburn River and across Gippsland to the coast. The neighbourhoods of Port Albert and Wilsons Promontory were explored, and the journey of some 1,500 miles (2,400 km) was completed along the coast to Melbourne

From 1854 to 1872, Mueller was a member of the Victorian Institute for the Advancement of Science, which later became the Philosophical Institute of Victoria. He was President of the Philosophical Institute in 1859 when it received a royal charter and became the Royal Society of Victoria. He was an active member of the society's "Exploration Committee" which established the Burke and Wills expedition of 1860. Mueller promoted the exploration of Australia, and as one of only two members of the Exploration Committee with any experience of exploration, he made several speeches to the society on the topic. He did not favour the selection of Burke as leader, but due to factionalism in the committee, he had little say in the establishment, provisioning, or composition of the exploration party.

Then, as a phytographic naturalist, he joined the expedition sent out under Augustus Gregory by the Duke of Newcastle, Secretary of State for the colonies. He explored the Victoria River and other portions of North Australia, was one of the four who reached Termination Lake in 1856, and accompanied Gregory's expedition overland to Moreton Bay. Mueller, for his part, found nearly 800 species in Australia new to science. He published in this year his Definitions of Rare or Hitherto Undescribed Australian Plants.

In 1857, Mueller applied for and was granted the degree of Doctor of Medicine of the University of Rostock; in 1883, he was awarded the Clarke Medal by the Royal Society of New South Wales.

Mueller was decorated by many foreign countries, including Germany, France, Spain, Denmark, and Portugal. He was appointed a fellow of the Royal Society in 1861, and knighted as Knight Commander of the Order of St Michael and St George in 1879. A list of his 'Orders, offices, affiliations and sundry honours' has been assembled. Many of his decorations were received in return for supplying zoological specimens to royal museums.

He published 11 volumes of Fragmenta phytographica Australiae (1862–1881), two volumes of the Plants of Victoria (1860–1865), and other books on the Eucalyptus, Myoporaceae, Acacia, and Salsolaceae, all profusely illustrated. He also co-operated in the production of George Bentham's Flora Australiensis. He described many novel plant species sent by botanists from other parts of Australia, notably Maurice william Holtze from the Northern Territory, and encouraged settlers to send plants to him. Many women contributed to his collections, including Louisa Atkinson and Sarah Brooks He took a leading part in promoting Australian exploration, especially the Burke and Wills expedition, which was the first to cross the continent, and in the various attempts to unravel the mystery which attended the fate of his fellow countryman Ludwig Leichhardt (1813–1848).

In the same year, he established the National Herbarium of Victoria, which can still be visited today. It has many plants from Australia and abroad, many of which were collected by Mueller. Also, his large private library was transferred to the government of Victoria in 1865 and is incorporated into the library of the herbarium in Melbourne.

He was the benefactor of Explorer Ernest Giles, the discoverer of Lake Amadeus and Kata Tjuta. Giles had originally wanted to name these Lake Mueller and Mt Ferdinand, but Mueller prevailed upon Giles to name them Lake Amadeus, after King Amadeus of Spain, and Mt Olga, after Queen Olga of Württemberg; both kings had granted him honours. In 1871, King Karl of Württemberg gave him the hereditary title of Freiherr, to mark his distinction in 'natural sciences generally and in particular for the natural history collections and institutions of Our Kingdom' He was then known as Baron Sir Ferdinand von Mueller.

In April 1873, Mueller had created the genus Guilfoylia and described william Guilfoyle as "distinguished as a collector [who] evidenced great ardour" and held high hopes for his collecting ability. Mueller's opinion changed when Guilfoyle was appointed to take his place as Director of the Botanic Gardens in July 1873. He accused Guilfoyle of being a "nurseryman [with] no claims to scientific knowledge whatever" and of getting the job due to being related to the wife of the responsible minister. Mueller subsequently abolished Guilfoylia as part of the genus of Cadellia in his botanical census of 1882. Guilfoyle went on to landscape the gardens in an aesthetic and pleasing style welcomed by most Melburnians.

The Mueller Medal has been awarded since 1904 by the Australian and New Zealand Association for the Advancement of Science to "a scientist who is the author of important contributions to anthropological, botanical, geological or zoological science, preferably with special reference to Australia".