When IOC president Jacques Rogge – who will retire after 12 years in the role on Tuesday – announced the winning city, the Tokyo delegation jumped to their feet in celebration and waved the Japan flag.
A number of them were overcome with emotion and wept, after two years of intense lobbying.
“I would like to thank everyone in the Olympic movement and we will host a wonderful Olympic Games,” a delighted Prime Minster Shinzo Abe said.
Bid leader Tsunekazu Takeda added: “It is a great honour that Tokyo has been chosen. The first thing I will do when I return is to thank all of Japan.”
London mayor Boris Johnson sent “huge congratulations to Tokyo for winning the honour of hosting the greatest sporting spectacular on the planet”.
“I am sure that, like London, your great city will put on an extraordinary event. This is a magical moment of celebration to savour before the years of hard work ahead.
The decision means Tokyo will become the first Asian city to host the Games twice.
They had previously been awarded the event in 1940, but the Games were cancelled because of World War II.
The success of the bid followed a personal address from the Japanese prime minister during the presentation stage, in which he allayed fears over the crippled Fukushima nuclear plant 150 miles (240km) from the city by saying: “It has never done, and will never do any damage to Tokyo.”
The plant had been leaking radiation after an earthquake and tsunami hit the north-east of the country in 2011.
The presentation also revealed the role sport had played in boosting the country in the past two years and pointed out that no Japanese athlete had failed a drugs test at an Olympics or Paralympics.
It added sponsorship would reach record levels and 10 new permanent sports venues would be constructed, including the Olympic Stadium, which will be finished by 2019 in time to host the Rugby World Cup.