Fireworks AI DevRel Head Sophia Yang Demonstrates Glimmer Model Training and Performance on Kalshi Prediction Markets
Sophia Yang, Head of DevRel at Fireworks AI, presents a video showing how to train the 30B open-source model Glimmer to outperform Jev in Kalshi prediction markets. The video details training on approximately 6,500 settled events and testing on 8,765 subsequent ones using K-way cross-entropy with LoRA. Claimed accuracy for tuned Glimmer is 64.8%, compared to 56.4% for the base model and 45.9% for Jev.
Sophia Yang, Head of DevRel at Fireworks AI, released a video detailing how her team trained Glimmer, a 30B parameter open-source model, to perform better than Jev in Kalshi prediction markets. This content is part of Fireworks AI's "Free Weights Club," offering a recipe and model that can be fine-tuned for free.
According to the video description, the experiment used about 6,500 settled Kalshi events for training and tested on 8,765 later events. The technical approach involved K-way cross-entropy loss with a single LoRA adapter, followed by temperature tuning.
Fireworks AI claims that the tuned Glimmer model achieved an accuracy of 64.8%, compared to 56.4% for the base model and 45.9% for Jev. Additionally, calibration was reported at 86.8% when the model was at least 80% sure, with serving speeds 2–3x faster than standard chat completions.
It is important to note that these performance metrics are vendor-reported and based on paper trading rather than actual capital operations. The video serves primarily as a technical demonstration and resource recommendation for readers interested in AI applications in prediction markets.