Looking at the attention maps: with the stable patch, LeWorldModel forgets the PushT object and focuses on the useless border. Make the patch unpredictable, and attention shifts back toward the task.