Toshodaiji was founded in the year 759 by Ganjin, a Chinese priest who was invited to Japan by the emperor in order to train priests and improve Japanese Buddhism.

Ganjin's tomb and the Miedo, a hall which stores a famous wooden statue of Ganjin, are located on the temple grounds. The statue, however, is displayed to the public only once a year on June 6, the anniversary of Ganjin's death.

The temple's lecture hall (kodo) was moved to Toshodaiji from the Nara Imperial Palace, and is now the only surviving building of the former palace. Toshodaiji's main hall (kondo) is currently being renovated until 2009.