Pokee AI Unveils 28B Model With 10 Million Token Context Window

Pokee-Isaac 28B achieves 10M token context, scoring 93.3 on RULER tests where baselines fail. Uses proprietary non-pure decoder architecture.

Woofun AI reports that AI startup Pokee AI has released Pokee-Isaac 28B, a model with 28 billion parameters and a 10 million token context window. This capacity is approximately ten times larger than the current mainstream limit of around 1 million tokens.

In self-evaluation, Isaac scored consistently around 95 points on the RULER Long Text Test from 256,000 to 4 million tokens, maintaining a score of 93.3 at 10 million tokens. Five baseline models failed to return usable results starting from 2 million tokens. Pokee attributes this performance to its self-developed "non-pure Decoder-only architecture," though the technical report has not disclosed specific details on attention mechanisms or memory allocation. Founded in 2024 by former Meta App Reinforcement Learning Lead Zhu Zheqing, Pokee AI previously raised a $12 million seed round led by Point72 Ventures, with participation from Qualcomm Ventures and Samsung NEXT.