A theory I saw on Reddit was that the spirits (Simon, Sven, Gunter, Santa and the possible others) are only free to roam independently in the Maze/crown town when Ice King is unconscious or asleep. But when Ice King is awake, it all gets messy and insane which is why Simon in “Betty” said he saw things as Ice King as weird abstract dream-like impressions. And why Simon described existing as Ice King as torture.
Its possible Simon’s soul is the only one that directly experiences Ice King’s insanity, because that’s his physical body and brain under the crown’s direct influence.
