a16z backs RL environment startup
a16z invests in Preference Model, which open-sourced Karotte, a framework for building reinforcement learning environments resistant to reward hacking. No financial terms disclosed.
ImportânciaLocalEvidênciaE2 não replicadaTratamentoRápido
a16z announced an investment in Preference Model, a company building reinforcement learning training environments for AI labs. The company this week open-sourced Karotte on GitHub, a framework designed to prevent models from exploiting shortcuts such as reading answer keys, modifying tests, or crashing graders during RL training.
Karotte's defenses include terminating stray processes before grading and rejecting malicious files. a16z states the framework has been hardened through over a million evaluation runs and red-teaming, though this claim comes from the investor's own announcement and has not been independently reproduced.
No investment amount, round stage, or valuation was disclosed.