Murray was born to william A. and Mary (née DePasquale) Murray, and grew up in Milford, Massachusetts. He was of Irish and Italian descent. A star athlete at the Milford High School, he excelled in football, ice hockey, and baseball. Upon graduation, Murray attended the College of the Holy Cross intending to play baseball; however, baseball practices and lab schedules conflicted; forcing him to give up baseball. He studied philosophy and English, earning a degree in humanities in 1940. Murray later attended Harvard Medical School. After graduating with his medical degree, Murray began his internship at the Peter Bent Brigham Hospital. During that time, he was inducted into the Medical Corps of the U.S. Army.

After his military Service, Murray completed his general surgical residency, and joined the surgical staff of the Peter Bent Brigham Hospital. He then went to New York to train in plastic surgery at New York and Memorial Hospitals, returning to the Brigham as a member of the surgical staff in 1951.

On December 23, 1954, Murray performed the world's first successful renal transplant between the identical Herrick twins at the Peter Bent Brigham Hospital (later Brigham and Women's Hospital), an operation that lasted five and a half hours. He was assisted by Dr. J. Hartwell Harrison and other noted Physicians. In Operating Room 2 of the Peter Bent Brigham Hospital, Murray transplanted a healthy kidney donated by Ronald Herrick into his twin brother Richard, who was dying of chronic nephritis. Richard lived for eight more years, following the operation. In 1959, Murray went on to perform the world's first successful allograft and, in 1962, the world's first cadaveric renal transplant.

Throughout the following years, Murray became an international leader in the study of transplantation biology, the use of immunosuppressive agents, and studies on the mechanisms of rejection. In the 1960s, top Scientists investigating immunosuppressive drugs sought to work with Murray. Together, they tailored the new drug Imuran (generic azathioprine) for use in transplants. The discovery of Imuran and other anti-rejection drugs, such as prednisone, allowed Murray to carry out transplants from unrelated donors. By 1965, the survival rates after receiving a kidney transplant from an unrelated donor exceeded 65%.

As a Harvard Medical School faculty member, Murray trained Physicians from around the world in transplantation and reconstructive surgery, frequently performing surgeries in developing countries. In his 20 years as Director of the Surgical Research Laboratory at Harvard and the Peter Bent Brigham Hospital, he inspired others who became Leaders in transplantation and biology throughout the world. He served as chief plastic surgeon at the Peter Bent Brigham (which later became Brigham and Women's Hospital) until 1986. He also served as chief plastic surgeon at Children's Hospital Boston from 1972–85, retiring as professor of Surgery Emeritus in 1986 from Harvard Medical School.

In 1990, he was honored with the Nobel Prize in Physiology or Medicine for his pioneering work in organ transplantation.

Murray was elected as a member of the National Academy of Sciences and as a regent of the American College of Surgeons. He received the American Surgical Association's Medal for Distinguished Service to Surgery, the American Academy of Arts and Sciences' Francis Amory Prize, the American Association of Plastic Surgeons' Honorary Award and Clinician of the Year Award, and the National Kidney Foundation's Gift of Life Award. He was named one of the 350 most outstanding citizens, representing the medical profession, for the City of Boston's 350th anniversary. In 1996, he was appointed Academician of the Pontifical Academy of Sciences in the Vatican.

In 2001, Murray published his autobiography, Surgery Of The Soul: Reflections on a Curious Career.

Murray died on November 26, 2012, aged 93. He suffered a stroke at his suburban Boston home on Thanksgiving and died at Brigham and Women's Hospital, the very hospital where he had performed the first organ transplant operation.