Spencer was born in Derby, England, on 27 April 1820, the son of william George Spencer (generally called George). Spencer's father was a religious dissenter who drifted from Methodism to Quakerism, and who seems to have transmitted to his son an opposition to all forms of authority. He ran a school founded on the progressive teaching methods of Johann Heinrich Pestalozzi and also served as Secretary of the Derby Philosophical Society, a scientific society which had been founded in 1783 by Erasmus Darwin, the grandfather of Charles Darwin.

In essence Spencer's philosophical vision was formed by a combination of deism and positivism. On the one hand, he had imbibed something of eighteenth century deism from his father and other members of the Derby Philosophical Society and from books like George Combe's immensely popular The Constitution of Man (1828). This treated the world as a cosmos of benevolent design, and the laws of nature as the decrees of a 'Being transcendentally kind.' Natural laws were thus the statutes of a well governed universe that had been decreed by the Creator with the intention of promoting human happiness. Although Spencer lost his Christian faith as a teenager and later rejected any 'anthropomorphic' conception of the Deity, he nonetheless held fast to this conception at an almost sub-conscious level. At the same time, however, he owed far more than he would ever acknowledge to positivism, in particular in its conception of a philosophical system as the unification of the various branches of scientific knowledge. He also followed positivism in his insistence that it was only possible to have genuine knowledge of phenomena and hence that it was idle to speculate about the nature of the ultimate reality. The tension between positivism and his residual deism ran through the entire System of Synthetic Philosophy.

Both as an adolescent and as a young man, Spencer found it difficult to settle to any intellectual or professional discipline. He worked as a civil Engineer during the railway boom of the late 1830s, while also devoting much of his time to writing for provincial journals that were nonconformist in their religion and radical in their politics. From 1848 to 1853 he served as sub-editor on the free-trade journal The Economist, during which time he published his first book, Social Statics (1851), which predicted that humanity would eventually become completely adapted to the requirements of living in society with the consequential withering away of the state.

The second objective of the Synthetic Philosophy was to show that these same laws led inexorably to progress. In contrast to Comte, who stressed only the unity of scientific method, Spencer sought the unification of scientific knowledge in the form of the reduction of all natural laws to one fundamental law, the law of evolution. In this respect, he followed the model laid down by the Edinburgh publisher Robert Chambers in his anonymous Vestiges of the Natural History of Creation (1844). Although often dismissed as a lightweight forerunner of Charles Darwin's The Origin of Species, Chambers' book was in reality a programme for the unification of science which aimed to show that Laplace's nebular hypothesis for the origin of the solar system and Lamarck's theory of species transformation were both instances (in Lewes' phrase) of 'one magnificent generalisation of progressive development.' Chambers was associated with Chapman's salon and his work served as the unacknowledged template for the Synthetic Philosophy.

Spencer greatly influenced literature and rhetoric. His 1852 essay, "The Philosophy of Style", explored a growing trend of formalist approaches to writing. Highly focused on the proper placement and ordering of the parts of an English sentence, he created a guide for effective composition. Spencer aimed to free prose writing from as much "friction and inertia" as possible, so that the reader would not be slowed by strenuous deliberations concerning the proper context and meaning of a sentence. Spencer argued that Writers should aim "To so present ideas that they may be apprehended with the least possible mental effort" by the reader.

The last decades of Spencer's life were characterised by growing disillusionment and loneliness. He never married, and after 1855 was a perpetual hypochondriac who complained endlessly of pains and maladies that no physician could diagnose. By the 1890s his readership had begun to desert him while many of his closest friends died and he had come to doubt the confident faith in progress that he had made the center-piece of his philosophical system. His later years were also ones in which his political views became increasingly conservative. Whereas Social Statics had been the work of a radical democrat who believed in votes for women (and even for children) and in the nationalisation of the land to break the power of the aristocracy, by the 1880s he had become a staunch opponent of female suffrage and made Common cause with the landowners of the Liberty and Property Defence League against what they saw as the drift towards 'socialism' of elements (such as Sir william Harcourt) within the administration of william Ewart Gladstone – largely against the opinions of Gladstone himself. Spencer's political views from this period were expressed in what has become his most famous work, The Man Versus the State.

Spencer first articulated his evolutionary perspective in his essay, 'Progress: Its Law and Cause', published in Chapman's Westminster Review in 1857, and which later formed the basis of the First Principles of a New System of Philosophy (1862). In it he expounded a theory of evolution which combined insights from Samuel Taylor Coleridge's essay 'The Theory of Life' – itself derivative from Friedrich von Schelling's Naturphilosophie – with a generalisation of von Baer's law of embryological development. Spencer posited that all structures in the universe develop from a simple, undifferentiated, homogeneity to a complex, differentiated, heterogeneity, while being accompanied by a process of greater integration of the differentiated parts. This evolutionary process could be found at work, Spencer believed, throughout the cosmos. It was a universal law, that was applying to the stars and the galaxies as much as to biological organisms, and to human social organisation as much as to the human mind. It differed from other scientific laws only by its greater generality, and the laws of the special sciences could be shown to be illustrations of this principle.

In 1858 Spencer produced an outline of what was to become the System of Synthetic Philosophy. This immense undertaking, which has few parallels in the English language, aimed to demonstrate that the principle of evolution applied in biology, psychology, sociology (Spencer appropriated Comte's term for the new discipline) and morality. Spencer envisaged that this work of ten volumes would take twenty years to complete; in the end it took him twice as long and consumed almost all the rest of his long life.

Spencer argued that both these theories were partial accounts of the truth: repeated associations of ideas were embodied in the formation of specific strands of brain tissue, and these could be passed from one generation to the next by means of the Lamarckian mechanism of use-inheritance. The Psychology, he believed, would do for the human mind what Isaac Newton had done for matter. However, the book was not initially successful and the last of the 251 copies of its first edition was not sold until June 1861.

In post-1863-Uprising Poland, many of Spencer's ideas became integral to the dominant fin-de-siècle ideology, "Polish Positivism". The leading Polish Writer of the period, Bolesław Prus, hailed Spencer as "the Aristotle of the nineteenth century" and adopted Spencer's metaphor of society-as-organism, giving it a striking poetic presentation in his 1884 micro-story, "Mold of the Earth", and highlighting the concept in the introduction to his most universal novel, Pharaoh (1895).

Spencer in his book Principles of Biology (1864), proposed a pangenesis theory that involved "physiological units". These hypothetical hereditary units were similar to Darwin's gemmules.

Spencer's reputation among the Victorians owed a great deal to his agnosticism. He rejected theology as representing the 'impiety of the pious.' He was to gain much notoriety from his repudiation of traditional religion, and was frequently condemned by religious thinkers for allegedly advocating atheism and materialism. Nonetheless, unlike Thomas Henry Huxley, whose agnosticism was a militant creed directed at 'the unpardonable sin of faith' (in Adrian Desmond's phrase), Spencer insisted that he was not concerned to undermine religion in the name of science, but to bring about a reconciliation of the two. The following argument is a summary of Part 1 of his First Principles (2nd ed 1867).

While most Philosophers fail to achieve much of a following outside the academy of their professional peers, by the 1870s and 1880s Spencer had achieved an unparalleled popularity, as the sheer volume of his sales indicate. He was probably the first, and possibly the only, Philosopher in history to sell over a million copies of his works during his own lifetime. In the United States, where pirated editions were still commonplace, his authorised publisher, Appleton, sold 368,755 copies between 1860 and 1903. This figure did not differ much from his sales in his native Britain, and once editions in the rest of the world are added in the figure of a million copies seems like a conservative estimate. As william James remarked, Spencer "enlarged the imagination, and set free the speculative mind of countless doctors, Engineers, and lawyers, of many physicists and chemists, and of thoughtful laymen generally." The aspect of his thought that emphasised individual self-improvement found a ready audience in the skilled working class.

Spencerian views in 21st century circulation derive from his political theories and memorable attacks on the reform movements of the late 19th century. He has been claimed as a precursor by libertarians and anarcho-capitalists. Economist Murray Rothbard called Social Statics "the greatest single work of libertarian political philosophy ever written." Spencer argued that the state was not an "essential" institution and that it would "decay" as voluntary market organisation would replace the coercive aspects of the state. He also argued that the individual had a "right to ignore the state." As a result of this perspective, Spencer was harshly critical of patriotism. In response to being told that British troops were in danger during the Second Afghan War (1878-1880) he replied: "When men hire themselves out to shoot other men to order, asking nothing about the justice of their cause, I don't care if they are shot themselves."

By the 1880s he was denouncing "the new Toryism" (that is, the "social reformist wing" of the Liberal party – the wing to some extent hostile to Prime Minister william Ewart Gladstone, this faction of the Liberal party Spencer compared to the interventionist "Toryism" of such people as the former Conservative party Prime Minister Benjamin Disraeli). In The Man versus the State (1884), he attacked Gladstone and the Liberal party for losing its proper mission (they should be defending personal liberty, he said) and instead promoting paternalist social legislation (what Gladstone himself called "Construction" – an element in the modern Liberal party that he opposed). Spencer denounced Irish land reform, compulsory education, laws to regulate safety at work, prohibition and temperance laws, tax funded libraries, and welfare reforms. His main objections were threefold: the use of the coercive powers of the government, the discouragement given to voluntary self-improvement, and the disregard of the "laws of life." The reforms, he said, were tantamount to "socialism", which he said was about the same as "slavery" in terms of limiting human freedom. Spencer vehemently attacked the widespread enthusiasm for annexation of colonies and imperial expansion, which subverted all he had predicted about evolutionary progress from 'militant' to 'industrial' societies and states.

Though Spencer made some valuable contributions to early sociology, not least in his influence on structural functionalism, his attempt to introduce Lamarckian or Darwinian ideas into the realm of sociology was unsuccessful. It was considered by many, furthermore, to be actively dangerous. Hermeneuticians of the period, such as Wilhelm Dilthey, would pioneer the distinction between the natural sciences (Naturwissenschaften) and human sciences (Geisteswissenschaften). In the United States, the Sociologist Lester Frank Ward, who would be elected as the first President of the American Sociological Association, launched a relentless attack on Spencer's theories of laissez-faire and political ethics. Although Ward admired much of Spencer's work he believed that Spencer's prior political biases had distorted his thought and had led him astray. In the 1890s, Émile Durkheim established formal academic sociology with a firm emphasis on practical social research. By the turn of the 20th century the first generation of German Sociologists, most notably Max Weber, had presented methodological antipositivism. However, it should be noted that Spencer's theories of laissez-faire, survival-of-the-fittest and minimal human interference in the processes of natural law had an enduring and even increasing appeal in the social science fields of economics and political science, and one Writer has recently made the case for Spencer's importance for a sociology that must learn to take Energy in society seriously.

Spencer developed an all-embracing conception of evolution as the progressive development of the physical world, biological organisms, the human mind, and human culture and societies. As a polymath, he contributed to a wide range of subjects, including ethics, religion, anthropology, economics, political theory, philosophy, literature, astronomy, biology, sociology, and psychology. During his lifetime he achieved tremendous authority, mainly in English-speaking academia. "The only other English Philosopher to have achieved anything like such widespread popularity was Bertrand Russell, and that was in the 20th century." Spencer was "the single most famous European intellectual in the closing decades of the nineteenth century" but his influence declined sharply after 1900: "Who now reads Spencer?" asked Talcott Parsons in 1937.

In 1902, shortly before his death, Spencer was nominated for the Nobel Prize for literature. He continued writing all his life, in later years often by dictation, until he succumbed to poor health at the age of 83. His ashes are interred in the eastern side of London's Highgate Cemetery facing Karl Marx's grave. At Spencer's funeral the Indian nationalist leader Shyamji Krishnavarma announced a donation of £1,000 to establish a lectureship at Oxford University in tribute to Spencer and his work.

Despite his reputation as a Social Darwinist, Spencer's political thought has been open to multiple interpretations. His political philosophy could both provide inspiration to those who believed that individuals were masters of their fate, who should brook no interference from a meddling state, and those who believed that social development required a strong central authority. In Lochner v. New York, conservative justices of the United States Supreme Court could find inspiration in Spencer's writings for striking down a New York law limiting the number of hours a baker could work during the week, on the ground that this law restricted liberty of contract. Arguing against the majority's holding that a "right to free contract" is implicit in the due process clause of the Fourteenth Amendment, Oliver Wendell Holmes Jr. wrote: "The Fourteenth Amendment does not enact Mr. Herbert Spencer's Social Statics." Spencer has also been described as a quasi-anarchist, as well as an outright anarchist. Marxist theorist Georgi Plekhanov, in his 1909 book Anarchism and Socialism, labelled Spencer a "conservative Anarchist."

Politics in late Victorian Britain moved in directions that Spencer disliked, and his arguments provided so much ammunition for conservatives and individualists in Europe and America that they are still in use in the 21st century. The expression 'There is no alternative' (TINA), made famous by Prime Minister Margaret Thatcher, may be traced to its emphatic use by Spencer.

However, as Bertrand Russell stated in a letter to Beatrice Webb in 1923, this formulation has problems: 'I don't know whether [Spencer] was ever made to realise the implications of the second law of thermodynamics; if so, he may well be upset. The law says that everything tends to uniformity and a dead level, diminishing (not increasing) heterogeneity'.

Spencer's ideas became very influential in China and Japan largely because he appealed to the reformers' Desire to establish a strong nation-state with which to compete with the Western powers. His thought was introduced by the Chinese scholar Yen Fu, who saw his writings as a prescription for the reform of the Qing state. Spencer also influenced the Japanese Westernizer Tokutomi Soho, who believed that Japan was on the verge of transitioning from a "militant society" to an "industrial society," and needed to quickly jettison all things Japanese and take up Western ethics and learning. He also corresponded with Kaneko Kentaro, warning him of the dangers of imperialism. Savarkar writes in his Inside the Enemy Camp, about reading all of Spencer's works, of his great interest in them, of their translation into Marathi, and their influence on the likes of Tilak and Agarkar, and the affectionate sobriquet given to him in Maharashtra – Harbhat Pendse.