We need to understand the subject of Gan Eden of the Earth, where the neshomas go up to when they are pure, specifically. It is known that the four rivers that go out from Gan Eden are literal, that there is a special place on the globe of this physical earth on the Equator, 32 degrees distance from Yerushalayim, etc., and there is the place of this Gan and the river goes out from Eden to water the Gan, and from there it divides, etc. The fact that they didn't find it on the face of the earth, and they don't see the flaming sword, etc., is because this Gan is in the dimension of Divine spirituality. As is known, the ground (karka) of this Gan is higher even than the four spiritual fundemental elements of fire, air, water, and dirt which are intermingled with good and evil, etc.
But the spirituality of the four fundamental elements makes the ground of this Gan in a way which has no admixture of evil at all, because this place is separated in spirituality from all admixture of evil, like the body of Adam Harishon, which is made from this dirt in the ground of the Gan, etc., and so if he wouldn't have sinned with the Tree of Knowledge, etc. Because it was all on the level of utmost Divine holiness, only that he sinned with the Tree of Knowledge of Good and Evil and he wasn't able to stand in Gan Eden, as it says "He expelled the man".
Maamarim of the Mitteler Rebbe, Vayikra II 502-3