Splitting Robot State into Tokens in GR00T N1.6

Question

GR00T N1.6 encodes the robot's whole joint state as one token, while the camera images and the instruction get many. For the ALLEX humanoid that means 48 numbers squeezed into a single token. Does the policy do better if the body gets more tokens?

What I did

• Split the 48 joint values into K equal-sized random groups, zero-padded each group to the same length, and passed every group through the existing state encoder, so the model sees K state tokens. Everything else in the model stays the same and no parameters are added.
• Trained GR00T N1.6 on three ALLEX manipulation tasks in Isaac Sim (cube into box, cube stacking, cylinder into tube) for K from 1 to 48, with three random groupings for each K between 3 and 15.
• Evaluated every model at three object offsets, 70 episodes × 3 runs each.

Result

With 11 or 15 tokens, mean success rose from 35.8% to about 41.5%, with little spread across groupings (±0.6 and ±0.9 points). More tokens is not simply better: 3 tokens did slightly worse than the baseline and 5 to 9 tokens stayed close to it. One token per joint (48) scored highest at 44.0%, but that is a single configuration I have not repeated yet.

What didn't help

Two follow-ups did not beat random groups. Choosing the groups by mutual information between joints or by body part gave about 42% at best, the same as random groups. Giving each part its own encoder, or conditioning a shared encoder on the part with FiLM, made results worse (32.8% and 33.7% against 41.6%).