Ossietzky was born in Hamburg, the son of Carl Ignatius von Ossietzky (1848–1891), a Protestant from Upper Silesia, and Rosalie (née Pratzka), a devout Catholic who wanted her son to enter Holy Orders, and become a priest or monk. His Father worked as a stenographer in the office of a Lawyer and senator, but died when Carl was two years old. Ossietzky was baptized as a Roman Catholic in Hamburg on 10 November 1889, and confirmed in the Lutheran Hauptkirche St Michaelis on 23 March 1904.

Despite his failure to finish Realschule (a form of German secondary school), Ossietzky succeeded in embarking on a career in journalism, with the topics of his articles ranging from theatre criticism to feminism and the problems of early motorization. He later said that his opposition to German militarism during the final years of the German Empire under Wilhelm II led him, as early as 1913, to become a pacifist.

That year, he married Maud Lichfield-Wood, a Mancunian suffragette, born a British colonial officer's daughter and the great granddaughter of an Indian Princess in Hyderabad. They had one daughter, Rosalinde. During World War I, Ossietzky was drafted much against his will into the Army, and his experiences during the war where he was appalled by the Carnage of the war confirmed him in his pacifism. During the Weimar Republic (1919–33), his political commentaries gained him a reputation as a fervent supporter of democracy and a pluralistic society.

In the 1920s, Ossietzky became one of the Leaders of the "homeless left", centered on the newspaper Die Weltbühne, which rejected Communism but found the Social Democrats too inclined to compromise with the old order.

In 1921, the German government founded the Arbeits-Kommandos (work squads) led by Major Bruno Ernst Buchrucker. Officially a labor group intended to assist with civilian projects, in reality they were used by Germany to exceed the limits on troop strength set by the Treaty of Versailles.

In this regard, Ossietzky at Die Weltbühne helped to publish a statistical study in 1923, showing that German judges were inclined to impose extremely harsh sentences on those who broke laws in the name of the left, while imposing very lenient sentences on those who committed violence in the name of the right. He often drew a contrast between the fate of the Social Democrat Felix Fechenbach who was imprisoned after a questionable trial for publishing secret documents showing that Reich was responsible for World War I and that of the Navy Captain Hermann Ehrhardt of the Freikorps whose men occupied Berlin during the Kapp Putsch, killed several hundred civilians and was never tried for his actions. At the same time, Ossietzky was often critical of those republicans who claimed to believe in democracy without actually knowing what democracy meant.

Ossietzy was especially critical of the Reichsbanner Schwarz-Rot-Gold (Reich Banner Black-Red-Gold), the paramilitary group set up by the Social Democrats to defend democracy. Ossietzky wrote in 1924:

In 1927, he succeeded Kurt Tucholsky as editor-in-chief of the periodical Die Weltbühne. In 1932, he supported Ernst Thälmann's candidacy for the German presidency, though still a critic of the actual policy of the German Communist Party and the Soviet Union.

In 1929 Walter Kreiser, one of the Writers for Die Weltbühne, published an expose of the training of a special air unit of the Reichswehr, referred to as Abteilung M ("M Battalion"), which was secretly training in Germany and in Soviet Russia, in violation of Germany's agreements under the Treaty of Versailles. Kreiser and Ossietzky, the paper's Editor, were questioned by a magistrate of the Supreme Court about the article later that year, and were finally indicted in early 1931 for "treason and espionage," the assertion being that they had drawn international attention to state affairs which the state had purposefully attempted to keep secret. The arrests were widely seen at the time as an effort to silence Die Weltbühne, which had been a vocal critic of the Reichswehr's policies and secret expansion.

Ossietzky continued to be a constant warning voice against militarism and Nazism. In 1932, he published an article in which he stated:

When in January 1933 Adolf Hitler was appointed Chancellor, the Nazi dictatorship began, but even then, Ossietzky was one of a very small group of public figures who continued to speak out against the Nazi Party. On 28 February 1933, after the Reichstag fire, he was arrested and held in so-called protective custody in Spandau prison. Wilhelm von Sternburg, one of Ossietzky's biographers, surmises that if Ossietzky had had a few more days, he would surely have joined the vast majority of Writers who fled the country. In short, Ossietzky underestimated the speed with which the Nazis would go about ridding the country of unwanted political opponents. He was detained afterwards at the Esterwegen concentration camp near Oldenburg, among other camps. Throughout his time in the concentration camps, Ossietzky was mercilessly mistreated by the guards while being deprived of food. In November 1935, when a representative of the International Red Cross visited Ossietzky, he reported that he saw "a trembling, deadly pale something, a creature that appeared to be without feeling, one eye swollen, teeth knocked out, dragging a broken, badly healed leg . . . a human being who had reached the uttermost limits of what could be borne."

Ossietzky's international rise to fame began in 1936 when, already suffering from serious tuberculosis, he was awarded the 1935 Nobel Peace Prize. The government had been unable to prevent this, but refused to release him to travel to Oslo to receive the prize. In an act of civil disobedience, after Hermann Göring prompted him to decline the prize, Ossietzky issued a note from the hospital saying that he disagreed with the authorities who had stated that by accepting the prize he would cast himself outside the deutsche Volksgemeinschaft (community of German people):

In May 1936 he was sent to the Westend hospital in Berlin-Charlottenburg because of his tuberculosis, but under Gestapo surveillance. He died in the Nordend hospital in Berlin-Pankow, still in police custody, on 4 May 1938, of tuberculosis and from the after-effects of the abuse he suffered in the concentration camps.

In 1963, East German television produced the film Carl von Ossietzky about Ossietzky's life, starring Hans-Peter Minetti in the title role. Von Ossietzky was portrayed in the comic series Berlin by Jason Lutes. The narrative based on Ossietzky, The Man Without a Party by Richard Tres, was presented as an ebook in 2018.

In 1991, the University of Oldenburg was renamed Carl von Ossietzky University of Oldenburg in his honor. Carl von Ossietzky's daughter, Rosalinde von Ossietzky-Palm, took part in the formal ceremony, accompanied by then Prime Minister of Lower Saxony Gerhard Schröder.

In 1992, Ossietzky's 1931 conviction was upheld by Germany's Bundesgerichtshof (Federal Court of Justice), applying the law as it stood in 1931.

Supporters of convicted Nobel Prize–winning Chinese dissident Liu Xiaobo have compared him to Ossietzky, both being prevented by the authorities from accepting their awards, and both having died while in custody. The International League for Human Rights (ILHR) awards an annual Carl von Ossietzky Medal "to honor citizens or initiatives that promote basic human rights."