Japanese xenophobia should be understood properly prior to jumping to any reactive conclusions. Xenophobia is not really racism, although it can occasionally cross the line in Japan and both are occasionally evidenced. For Japanese society any xenophobia is more of a fear of the unknown and a fear of being overpowered by that which is unknown. In all honesty, many people in Japan are rather intimidated by other countries and also their relatively large people. It may sound silly, but it really does stem from the most basic things like physical size.
Due in part to historically taking the offense on xenophobia, the Japanese government has historically made some rather unwise choices internationally. The results of such choices have made the nation feel somewhat guilty and also somewhat worried about belated retaliation from its closer neighbors, in addition to simply feeling defeated. To go from a state of megalomania to a state of apocalyptic defeat is obviously rather shocking and, as a result, Japan is now much more cautious about its role on the world stage. The nation of Japan was also occupied after WWII and essentially had its entire culture exposed before a relatively judgmental Western perspective. Apparently as a result, some people in Japan also seem to fear exposure and being misunderstood or judged for their lifestyles.
Many things simply are drastically different in Western and Eastern culture. Japan is even quite unique in Eastern culture. To say one way of doing things is correct and one is incorrect would likely be primarily based on the cultural background of a viewer and thus it would be an inaccurate assessment. There are many things considered normal in most of Western society that are not generally considered acceptable in Japanese society, and also certainly many things in Japanese society which would not be acceptable in most of Western society. It is easy to understand why it could be somewhat more challenging to open up completely if there is a history of being judged for some of the most basic aspects of a society.
In addition to concerns about average physical size and a historical tendency for Western cultures to misunderstand and judge Japanese society, there are also basic differences in the general psychology of various societies. Japan has what is most likely the most obedient modern society in the world. There is no modern history of revolution and there are no riots in the streets. People are generally quite polite and Tokyo, although the most populated metropolis in the world, is possibly the safest city for a child to be in at any time of day or night. Purse snatching and the like are practically unheard of and are certainly uncommon. People will stand in herds at a street corner and wait for the light on the crosswalk to change to green, regardless of there not being a car in sight. It is understandable that there could be some concern over people from societies in which the cars don’t even stop at stoplights, as is common even in other parts of Asia.
In a way, Japan is like a clock and Japanese society generally seems to like to keep things running like clockwork. Thankfully the introduction and adoption of new ideas is part of what makes has made that clock run so well, and so Japanese society is still quite open to careful introductions. It just isn’t so game for any oversized unhygienic elephants to come trampling through its delicate clockwork.