We need to understand the subject of Gan Eden that is on the Earth, for it states "Hashem Elokim planted a Gan in Eden to the East" and according to the simple understanding it means that there is a special place in this physical world which is the place of this Gan, as explained in the Zohar. And this place, even though it is in this physical world, nevertheless it is on the level of Divine spirituality. It can be said that this is the level of spirituality that is in the physicality of the earth. And it is known that in every physical thing there is a dimension of "spirituality that is in physicality", which means besides the spiritual life-force in it, which is the dimension of actual spirituality, we find in the physical itself that there is a dimension of spirituality in it, within which the spiritual life-force is enclothed, etc.
Now Adam Harishon's physical body was refined and purified from all impurity amd waste and, quite the opposite, he was glowing and luminescent, and the midrash says his heel darkened the globe of the sun, etc. And if he didn't sin with the Tree of Knowledge he would remain on the level of ultimate Divine spirituality.
Rebbe Rashab, Maamorim 5662 p.299