Not as small as my transformer running on an original Game Boy 😂 3k parameters! Obviously it can only say a few things, but it keeps coherence inside the sentence.

You can run a transformer in a Game Boy from 1989 :)

Moon Below grew out of DMGFormer, my earlier project proving that a real, integer-only transformer forward pass could run on the original Game Boy. DMGFormer proved the runtime. Moon Below turns it…