OpenAI's New Decisions API vs. JEV & Julia-1: Building a 10ms-to-Cloud Reflex Cascade in Coni

Over the past two weeks, we have been obsessed with a single architectural thesis: autonomous agents should not use slow, token-dripping chat models to make bounded routing decisions.

First, we wired Coni’s Go-backed CSP channels into Ollama 0.35’s System One API to run TypeSafe’s JEV causal decision models (nimble and tev1). A week later, when Ollama choked on SupersonicLabs’ bidirectional ModernBERT architecture, we built a native 24-layer Julia-1 engine directly on Apple Metal MLX, hitting 10 milliseconds per inference in pure Coni.

And now, the 800-pound gorilla has entered the room.

OpenAI just launched their public beta Decisions API (POST /v1/decisions), powered exclusively by a new model called gpt-6-luna. Naturally, headlines across Towards Data Science and Hacker News immediately asked the obvious question:

“Is OpenAI’s Decisions API the JEV killer?”

Since we already run both JEV (nimble) and Julia-1 natively in Coni—and we just built support for OpenAI’s /v1/decisions (libs/llm/examples/openai_decisions_api.coni)—we don’t have to guess. We can benchmark the schemas side-by-side, expose the sharpest edges of each approach, and build something completely new that neither camp is talking about: a Confidence-Gated Reflex Cascade (System 1.0 $\rightarrow$ System 1.5) that fuses 10ms local Apple Metal inference with multimodal cloud escalation.

1. What OpenAI Actually Shipped (POST /v1/decisions)

For years, if you wanted OpenAI to classify a support ticket or score a document against a rubric, you had two clunky options:

1. Structured Outputs / JSON Mode: Pay a massive generative model to emit syntactic JSON tokens { "department": "shipping" } character by character.
2. Logit Bias + top_logprobs Hacks: Force max_tokens=1, map your categories to single BPE token IDs, and pray the tokenizer didn’t split "billing" into two subwords.

OpenAI’s /v1/decisions throws both hacks away. Powered by gpt-6-luna, it takes a shared input (text, or up to 128 inline base64 images) and evaluates an array of typed questions in a single pass, returning calibrated probability distributions and confidence scores—about 10× faster than the standard Responses API.

Even better is the pricing model: $0.10 per 1M input tokens, and $0.00 for output tokens. You only pay for the evidence you send in.

It supports three bounded judgment types:

• predicate: A binary yes/no condition returning a probability in $[0, 1]$.
• choice: Unordered categorical selection from fixed {:value :description} options, returning choice, confidence, and probabilities.
• score: Ordered rubric evaluation across {:label :description} levels, returning a 0-indexed probability-weighted average score, confidence, and probabilities.

2. Schema Showdown: JEV / System One vs. OpenAI Decisions API

If you have been using JEV or our serve_systemone.coni daemon, OpenAI’s payload will look eerily familiar. Clearly, the industry is converging on the exact same cognitive primitive—with a few subtle schema traps.

In fact, when we first wired /v1/decisions into Coni, we immediately hit a classic schema gotcha: choice options use :value, whereas score levels require :label! Send :value inside a score rubric and OpenAI’s validator will silently reject the payload.

Here is how the three engines stack up in practice:

3. Writing a Clean OpenAI Decisions Client in Coni

In Coni (libs/llm/examples/openai_decisions_api.coni), we can express all three of OpenAI’s question types as tiny, composable functional data builders:

(require "libs/http/src/http.coni" :as http)
(require "libs/json/src/json.coni" :as json)
(require "libs/str/src/str.coni" :as str)

(def *decisions-endpoint* "https://api.openai.com/v1/decisions")
(def *default-model* "gpt-6-luna")

(defn q-predicate
  "Creates a binary yes/no predicate question."
  [q-name instructions]
  {:type "predicate"
   :name q-name
   :instructions instructions})

(defn q-choice
  "Creates a categorical choice question from {:value :description} maps."
  [q-name instructions choices]
  {:type "choice"
   :name q-name
   :instructions instructions
   :choices (into []
                  (map (fn [c] (if (map? c) c {:value (str c)}))
                       choices))})

(defn q-score
  "Creates an ordered rubric score question from {:label :description} maps."
  [q-name instructions levels]
  {:type "score"
   :name q-name
   :instructions instructions
   :levels levels})

Notice how clean it is to batch a :predicate, a :choice, and a :score against the same ticket in a single HTTP request, then index the returned :answers vector into a keyword map via reduce:

(defn decide! [input questions & [opts]]
  (let [api-key (or (:api-key opts) (sys-env-get "OPENAI_API_KEY"))
        payload {:model (or (:model opts) *default-model*)
                 :input input
                 :questions questions}
        raw     (http/fetch *decisions-endpoint*
                            {:method "POST"
                             :headers {"Authorization" (str "Bearer " api-key)
                                       "Content-Type"  "application/json"}
                             :body (json/stringify payload)})]
    (json/parse raw)))

(defn index-answers [response]
  (reduce (fn [acc ans]
            (assoc acc (keyword (:name ans)) ans))
          {}
          (or (:answers response) [])))

Running our triage suite (./coni libs/llm/examples/openai_decisions_api.coni) produces:

=== OpenAI Decisions API ( gpt-6-luna ) ===
Input Ticket: The monitor arrived with a cracked screen and won't power on. I need an urgent replacement before our client presentation tomorrow. 

1. Physical Damage?  -> P(damaged) = 0.96 [YES - Trigger RMA flow]
2. Routed Department -> shipping (confidence: 0.91)
3. Issue Severity    -> 1.88 / 2.0 (confidence: 0.86) [HIGH PRIORITY]

4. Why “OpenAI vs. JEV” is the Wrong Question

Everyone debating whether gpt-6-luna kills local JEV / Julia-1 models is missing the most important mathematical property of these APIs:

Decision models don’t just return an answer—they return calibrated :confidence and :probabilities distributions.

When you ask a traditional LLM to generate text, it doesn’t know what it doesn’t know; it happily hallucinates with complete syntactic confidence. When you ask a decision model like Julia-1 on local Apple Metal to classify an input, it tells you its exact epistemic certainty in 10 milliseconds.

Think about what that means for system architecture:

• Why pay 250ms of TLS + cloud network latency (and leak customer text to the cloud) for the 90% of routine decisions where a 144M-parameter local ModernBERT model on Apple Metal is 99.9% confident in 10ms?
• Conversely, why force a 144M text-only local model to guess when a ticket has low confidence (< 0.80), nuanced multi-lingual sarcasm, or an attached customer photo of a smashed package?

5. Something New: The Speculative Reflex Cascade (System 1.0 $\rightarrow$ System 1.5)

Daniel Kahneman popularized System 1 (fast, instinctive) vs. System 2 (slow, deliberative reasoning).

With both native Apple Metal Julia-1 and OpenAI gpt-6-luna available inside Coni, we can split System 1 itself into a two-stage autonomic nervous system:

• System 1.0 (Spinal Reflex — Local Metal Julia-1, 10ms, $0): Handles every text event first in Unified Memory.
• System 1.5 (Cortical Reflex — Cloud gpt-6-luna, ~200ms, Multimodal): Triggered only when System 1.0 reports epistemic uncertainty (confidence < 0.85) or when visual evidence (input_image) is attached!
• System 2.0 (Deep Reasoning — gpt-6 / qwen3 Agent): Invoked only when the decision cascade itself flags a complex workflow requiring multi-step tool synthesis.

Look at how effortlessly this Speculative Reflex Cascade reads in Coni:

(require "libs/llm/src/julia.coni" :as julia)

(defn multimodal-input
  "Wraps text and base64 PNGs into OpenAI's /v1/decisions message format."
  [text base64-images]
  [{:role "user"
    :content (into [{:type "input_text" :text text}]
                   (map (fn [img]
                          {:type "input_image"
                           :image_url (str "data:image/png;base64," img)})
                        base64-images))}])

(defn reflex-decide!
  "System 1.0 -> System 1.5 Speculative Decision Cascade.
   Evaluates locally on Apple Metal in 10ms first; escalates to OpenAI gpt-6-luna
   only on multimodal inputs or low local confidence (< threshold)."
  [local-model event spec & [threshold]]
  (let [min-conf (or threshold 0.85)
        images   (:images event)
        text     (:text event)]
    (if (not (empty? images))
      ;; 1. Multimodal fast-path -> Directly to OpenAI gpt-6-luna
      (let [ans (index-answers (decide! (multimodal-input text images) (:openai-q spec)))]
        {:tier :system-1.5-multimodal
         :result ans})
      ;; 2. Local 10ms Apple Metal Reflex (Julia-1)
      (let [local-ans (julia/predict-question local-model text (:julia-q spec))]
        (if (>= (:confidence local-ans) min-conf)
          {:tier :system-1.0-metal
           :latency-ms 10
           :result local-ans}
          ;; 3. Epistemic Escalation -> Cloud gpt-6-luna breaks the tie!
          (let [cloud-ans (index-answers (decide! text (:openai-q spec)))]
            {:tier :system-1.5-escalated
             :local-confidence (:confidence local-ans)
             :result cloud-ans}))))))

6. Going One Step Further: Hedged CSP Quorum Racing

What if you are trading, moderating live voice streams, or guarding a production deployment where both tail latency and subtle semantic accuracy matter?

Because Coni gives us native Go channels (chan) and goroutines (spawn), we don’t even have to run System 1.0 and System 1.5 sequentially. We can spawn a Parallel Hedged Race:

1. Goroutine A runs Julia-1 on Apple Metal and returns in 10ms.
2. If Julia-1’s confidence is $> 0.92$, we cancel/ignore the cloud call and commit the action immediately in 10ms.
3. Meanwhile, Goroutine B is already in-flight to gpt-6-luna. If Julia-1 was uncertain, gpt-6-luna is already 10ms into its network flight—and when it lands, we can compute a precision-weighted log-odds ensemble of both models’ probability distributions!

(defn combine-probabilities
  "Fuses local Julia-1 and cloud gpt-6-luna probabilities weighted by model confidence."
  [local-p local-conf cloud-p cloud-conf]
  (let [w-total (+ local-conf cloud-conf)]
    (/ (+ (* local-p local-conf)
          (* cloud-p cloud-conf))
       w-total)))

Notice the bonus superpower at step 5: even when the 10ms local model wins the race, letting the background gpt-6-luna goroutine finish asynchronously gives you free, zero-latency shadow evaluation to detect distribution drift and auto-curate fine-tuning datasets for your local GGUF model!

Verdict: Is gpt-6-luna the JEV Killer?

No—it’s the missing half of the System One stack.

OpenAI’s /v1/decisions validates everything the local JEV and Julia-1 community has been building: decisions are a first-class primitive distinct from text generation.

• When you need multimodal inspection (up to 128 images) or frontier zero-shot judgment on messy edge cases without paying output token costs, gpt-6-luna is unmatched.
• When you need 10ms deterministic latency, zero network jitter, 100% air-gapped privacy, and $0 marginal cost across thousands of concurrent CSP goroutines, our native Apple Metal Julia-1 and JEV engine in Coni reigns supreme.
• And when you wire both together into a Confidence-Gated Reflex Cascade in Coni, you get an agentic nervous system that thinks in 10 milliseconds and escalates only when it knows it needs a second opinion.

Check out the full runnable code in the Coni repository:

• OpenAI Decisions API Client & Demo: libs/llm/examples/openai_decisions_api.coni
• Native Metal Julia-1 Engine: libs/llm/src/julia.coni
• System One Swarm Showcases: libs/llm/examples/system_one/