Might as well ask why lava crabs swarmed when we tried to pull one out of a volcano? Is it the orb, or are we just made of meat.

[Who knows? But as soon as Bucky touches the orb, the simulation around them fades out much like it would if they were transported back to the station on a mission.]