Gandel was born in Australia in 1935, the son of Sam and Fay Gandel, Jewish immigrants from Poland. His parents operated a corsetry store on Collins Street in Melbourne. Gandel was educated at Melbourne High School. He has one sister, Eva Gandel Besen.

In the 1950s, Gandel took control of his parents' women's clothing Business (then named Sussan) in the 1950s and with his brother-in-law Marc Besen grew it into a chain of over 200 stores. In 1983, he bought the Chadstone Shopping Centre, selling Sussan to his brother-in-law in 1985 to concentrate on real estate. He subsequently built his own fortune in shopping centres.

Since 1978, Gandel and his wife have headed up Gandel Philanthropy, one of the largest philanthropic family funds in Australia. The foundation provides grants to charitable community organisations for Jewish and Australian causes.

In July 1983, the Myer Emporium sold the Chadstone Shopping Centre to the Gandel Group for A$37 million, which has since managed and developed the complex into Australia's biggest shopping mall which is now valued at over $3 billion. In July 1983, the Myer Emporium also sold the Northland Shopping Centre to the Gandel Group. Gandel's major assets are 50 per cent shares in Chadstone, and a 17 per cent stake in the listed Novion that owns the other half. Gandel also owns significant stakes in listed property companies, besides the stake in Novion, worth about A$800 million, and a 15 per cent stake in Sydney-based property group, Charter Hall, in which he invested A$151 million.

Gandel is married to Pauline; together they have had four children. John and Pauline Gandel live in Toorak, Melbourne. In 1990, Gandel was appointed an Officer of the Order of Australia in recognition of his Service to Business, commerce and to the community.

In May 2006, Gandel was awarded an honorary doctorate of philosophy from Tel Aviv University, the university's highest honor, for his "business standing, broad leadership roles and philanthropic support in Australia and Israel." In the same month Gandel inaugurated the Gandel Institute for Adult Jewish Learning at the Hebrew University of Jerusalem.

Gandel was targeted as a possible major donor to Israeli Politician Benjamin Netanyahu before the Likud's primary election in 2007, which he subsequently won.

Gandel's philanthropy has focused on Judaism, education, medical causes and the arts. In 2010 he donated A$7.5 million to the National Gallery of Australia; A$1 million to fight devastating bushfires in Victoria, recalling how their own home was destroyed by fire 20 years earlier, and A$1 million to Museums Victoria.

According to a January 2012 Herald Sun report, Gandel was the wealthiest man in Melbourne, with an estimated A$3.4 billion net worth. In 2012 Gandel sold his stake in the Northland Shopping Centre for A$455 million and announced an A$500 million hotel and office redevelopment at Chadstone in 2011. In 2006, Gandel bought a 50% stake in Plastro Irrigation Systems, an Israeli irrigation company. In May 2007, the majority owners of the company exercised a right they held and bought back these holdings.

In 2017, Gandel was named Companion (AC) of the Order of Australia for his Service to the community as a benefactor and supporter of a range of visual arts and cultural institutions, to youth education, medical and biotechnology innovation programs, to Business and to the advancement of philanthropic giving.