He was born Johann Georg Nicolai in Essingen, Rhenish Bavaria. In 1838, he immigrated to the United States with his Father and attended school in Cincinnati. He later moved to Illinois, where he edited the Pike County Free Press at Pittsfield, and became a political power in the state. Then he became assistant to the secretary of state of Illinois. While in this position, he met Abraham Lincoln and became his devoted adherent.

In 1861, Lincoln appointed Nicolay as his private secretary, which was the first official act of his new administration. Nicolay served in this capacity until Lincoln's death in 1865. Shortly before his assassination, Lincoln appointed Nicolay to a diplomatic post in France. After the death of the President, Nicolay became United States Consul at Paris, France (1865–69). For some time after his return to the United States, he edited the Chicago Republican. He was Marshal of the United States Supreme Court (1872–1887). In 1881, Nicolay wrote The Outbreak of the Rebellion.

Nicolay was a founding member of the Literary Society of Washington in 1874, according to a book about the society written by his daughter, Helen Nicolay. Both Nicolay and Hay were members of long standing in the society.

Nicolay and John Hay, who had worked alongside Nicolay as assistant secretary to Lincoln, collaborated on the official biography of the 16th President. It appeared in The Century Magazine serially from 1886 to 1890 and was then issued (1890–94) in book form as ten volumes, together with the two-volume Complete Works of Abraham Lincoln. The resulting biography is a definitive resource on Lincoln and his times. Nicolay and Hay also edited Lincoln's Works in twelve volumes (1905). Finally, Personal Traits of Abraham Lincoln was published by Helen Nicolay in 1912.

Poor health had forced Nicolay to resign as Marshal of the Supreme Court, and he suffered from a wide range of ailments in his final years. He lived with his spinster daughter, Helen Nicolay, at her home at 212 B Street SE in Washington, D.C. He died at home of unspecified causes on September 26, 1901. He was buried at Oak Hill Cemetery in the city.

In the 1992 documentary Lincoln, the German-born Nicolay is voiced by the Austrian-born actor Arnold Schwarzenegger. In the 1988 NBC mini-series Lincoln based on Gore Vidal's book, Nicolay is portrayed by actor Richard Travis. In the 2012 film Lincoln, Nicolay is portrayed by Jeremy Strong. In the 2017 documentary film The Gettysburg Address, Nicolay is portrayed by actor william Fichtner.