Fast Native LLM Inference in Coni: The Poem Generator

Sat, 25 Jul 2026 11:45:00 +0900

In our ongoing quest to make Coni the most capable, dependency-free, and relentlessly fun functional language out there, we’ve hit a massive milestone. We didn’t just write an AI script—we went fully native.

Today, we’re unveiling our first fully native AI example: The Poem Generator.

Using the highly capable Qwen 2.5 0.5b instruct model formatted as a raw GGUF binary, this new Coni script takes a theme, invokes a local LLM, computes a beautiful poem, and spits out characters at a shockingly fast rate of over 70 tokens per second (TPS).

Cgo on NicoLabs

Fast Native LLM Inference in Coni: The Poem Generator