C.N.R. Rao was born in Bangalore to Hanumantha Nagesa Rao and Nagamma Nagesa Rao. He was an only child, and his learned parents made an academic environment. He was well versed in Hindu literature from his mother and in English from his Father at an early age. He did not attend elementary school but was home-tutored by his mother, who was particularly skilled in arithmetic and Hindu literature. He entered middle school in 1940, at age six. Although he was the youngest in his class, he used to tutor his classmates in mathematics and English. He passed lower secondary examination (class VII) in first class in 1944. He was ten years old, and his Father rewarded him with four annas (twenty-five paisa). He attended Acharya Patashala high school in Basavanagudi, which made a lasting influence on his interest in chemistry. His Father enrolled him to a Kannada-medium course to encourage his mother tongue, but at home used English for all conversation. He completed secondary school leaving certificate in first class in 1947. He studied BSc at Central College, Bangalore. Here he developed his communication skills in English and also learnt Sanskrit. He obtained his bachelor's degree from Mysore University in 1951, in first class, and only at the age of seventeen. He initially thought of joining Indian Institute of Science (IISc) for a diploma or a postgraduate degree in chemical engineering, but a Teacher persuaded him to attend Banaras Hindu University. He obtained a master's in chemistry from BHU two years later. In 1953 he was granted a scholarship for PhD in IIT Kharagpur. But four foreign universities, MIT, Penn State, Columbia and Purdue also offered him financial support. He chose Purdue. His first research paper was published in the Agra University Journal of Research in 1954. He completed PhD in 1958, only after two years and nine months, at age twenty-four.

After completion of his graduate studies Rao returned to Bangalore in 1959 to take up a lecturing position, joining IISC and embarking on an independent research program. From 1963 to 1976 Rao accepted a permanent position in the Department of Chemistry at the Indian Institute of Technology Kanpur, during which time he was elected as a fellow of the Indian Academy of Sciences in 1964. He returned to IISc in 1976 to establish a solid state and structural chemistry unit. and became Director of the IISc from 1984 to 1994. At various points in his career Rao has taken appointments as a visiting professor at Purdue University, the University of Oxford, the University of Cambridge and University of California, Santa Barbara. He was the Jawaharlal Nehru Professor at the University of Cambridge and Professorial Fellow at the King's College, Cambridge during 1983–1984.

Rao is married to Indumati Rao in 1960. They have two children, Sanjay and Suchitra. His son Sanjay Rao is engaged in popularising science in Bangalore's schools. His daughter Suchitra is married to K.M. Ganesh, the Director of the Indian Institute of Science Education and Research (IISER) at Pune, Maharashtra. Rao is quite technophobic. He removed computers from his tables and never checks his email by himself. He also said that he uses the mobile phone only to talk to his wife.

Rao is currently the National Research Professor, Linus Pauling Research Professor and Honorary President of Jawaharlal Nehru Centre for Advanced Scientific Research, Bangalore which he founded in 1989. He was appointed Chair of the Scientific Advisory Council to the Indian Prime Minister in January 2005, a position which he had occupied earlier during 1985–89. He is also the Director of the International Centre for Materials Science (ICMS) and serves on the board of the Science Initiative Group.

Rao has been subject of a number of plagiarism scandals, allegations he refutes. In December 2011, an apology was issued to the journal Advanced Materials – a peer-reviewed publication – for a paper he co-authored with a junior research associate that contained text reproduced without attribution of other scientist's work. The PhD student assumed the responsibility for the incident and issued a formal apology. Later Rao offered a full withdrawal the article from the journal, however the Editor let the publication stay. Rao claimed to not be complicit in the incident plagiarism.

On 17 November 2013, at a press conference following the announcement of his Bharat Ratna, he called the Indian politicians "idiots" that caused a national outrage. He said, "Why the hell have these idiots [politicians] given so little to us despite what we have done. For the money that the government has given us we [scientists] have done much more." In his defence Rao insisted that he merely talked about the "idiotic" way the politicians ignore Investments for research funding in science.