How Macropoiesis now separates two kinds of machine judgement — and why the slow kind was never the right tool for the fast question.

Abstract

Since 2026-09-21 three classification steps on Macropoiesis — the nightly article-sentiment scoring behind the Context-Conditioned RSI, the earnings-day headline tags on stock pages, and the five-band framing analysis in The Spectrum — can be answered by a System One decision model instead of a generative language model. A System One model does not write text. It takes a piece of evidence and a set of typed questions and returns, for every option, a calibrated probability. Replayed over 6,410 historical article scorings, the decision model reproduced the generative model's directional verdict in 80.4 % of cases, agreed on the daily aggregate that actually reaches our database with a correlation of 0.965, and answered in a median of 238 milliseconds against 2.9 seconds. This note records the hypothesis, the method, the numbers, and the limits — including the cases where the new model is confidently wrong.

1. The wrong tool for a typed question

Much of what a research platform asks a language model to do is not writing. Is this headline bullish, bearish or neutral for the stock? Which of five emotional registers is this article written in? Should this item pass to the expensive stage of the pipeline? These are questions with a fixed, small set of answers. For two years the convenient way to ask them has been to prompt a generative model, instruct it to "reply in JSON only", and parse whatever comes back.

That convenience has a cost that compounds. A generative model answers a typed question by generating — token by token, sometimes reasoning aloud first — so a three-way label costs the same seconds as a paragraph. It returns a number it was asked to invent ("probability 0.85") rather than one it computed, so the confidence is decoration. And it occasionally returns prose where JSON was requested, which is why our codebase had accumulated regex repair, chunking work-arounds and "please, only the JSON" pleading in prompts.

Our own telemetry made the scale of this concrete. Over ten days, 38 % of all wall-clock time spent waiting on language models on this site was one job: scoring news articles into positive / neutral / negative for the Context-Conditioned RSI, 2,235 calls at a median of 2.9 s each. The output of every one of those calls was a 44-character JSON object.

2. Hypothesis

In the vocabulary of dual-process psychology, we had been using a System Two process — slow, deliberate, verbal — for a System One task: fast, pattern-matched, non-verbal. The hypothesis was that a model built for the second kind of judgement would match the first on direction, beat it on speed by more than an order of magnitude, and — because its probabilities are trained against outcomes rather than improvised — give us a usable notion of confidence for the first time.

A System One decision model, in the sense we use here, has three properties: it accepts a state (the evidence: a headline and excerpt) and one or more typed questions (a choice among named options, a score on an ordered scale, or a yes/no); it returns a probability distribution over the options, not text; and its probabilities are calibrated, meaning a stated 0.8 is right about 80 % of the time.

3. Method

We did not switch anything on. We replayed.

The site keeps a log of every classification it has asked a language model to make, together with the answer. For the sentiment task that log held 6,410 article scorings. Each was re-submitted to the decision model as a single three-way choice — positive, neutral, negative — carrying the same investor-reading guidance the generative prompt uses ("lower-than-expected losses is mildly positive; record profits but guidance cut is mixed, leaning negative; ignore human-interest framing"). The two answers were then compared on three levels:

1. Per article: does the most probable label agree?
2. Per persisted value: the pipeline stores only the signed score p(positive) − p(negative); how far apart are the two models on that number?
3. Per day: what reaches the RSI model is a weighted daily average over roughly thirty articles per symbol. Do the daily aggregates track each other?

Level 3 is the one that matters. Nobody trades on one headline's label; the feature is the average.

4. Results

Two things in that table are more interesting than the headline agreement.

First, the confidence means something. Where the decision model was sure, it agreed with the generative model 95 % of the time; where it was unsure, a coin flip. The generative model's self-reported "0.85" never separated those cases — it said 0.85 about almost everything. A calibrated number can be acted on: route the uncertain fifth of articles to a slower reviewer, trust the rest.

Second, the disagreements are mostly "neutral". Of 1,258 per-article disagreements, 950 were the decision model calling neutral something the generative model had called directional — partnership announcements, product expansions, "X pushes deeper into Y". Direct reversals of direction were rare, and confident reversals almost non-existent. The decision model is the more conservative reader.

5. Limitations

We adjudicated the forty largest disagreements by hand, and the generative model was right more often — about three to one. The pattern is instructive. The decision model reads literally. "RBC drops jaw-dropping price target on Dell" is bullish slang; the model read drops and returned negative at confidence 1.00. "Super Micro rallied on a stamp of approval, not on demand" is a sceptical headline; the model returned positive at 0.95. Rhetorical contrast, idiom and irony are precisely where a fast pattern-matcher fails, and — this is the uncomfortable part — it sometimes fails confidently. Four articles in 6,410 is a small number, but it is not zero, and a calibrated model that is wrong at 1.00 is a reminder that calibration is a population property, not a guarantee per item.

Three further caveats. The comparison target is itself a model, not ground truth; agreement measures consistency, not correctness. The daily correlation of 0.965 was computed over corpus-days of many articles; symbol-days of thirty will be noisier. And any change of scorer is a structural break in a time series: every row the Context-Conditioned RSI persists records which model produced it, and analyses across the changeover date must treat the two vintages separately. That disclosure was already part of the RSI methodology; it now has a second entry.

6. What changes for readers

Nothing visible, at first — which is the point. The three sites run behind flags that default to off and are switched on one at a time, starting with the nightly sentiment job, after the evidence above. When they are on: the earnings-day headline tags on a stock page arrive in a single request instead of one per headline (roughly ten seconds of progressive fill becomes well under one); The Spectrum's five-band framing is computed in one concurrent pass instead of four sequential chunks that occasionally overran and fell back to a lexicon; and the nightly RSI job finishes in minutes rather than the better part of an hour, with the tooltip on each tag now carrying the model's own confidence rather than a sentence it composed.

Generative models keep every job that is actually writing — the Tribune, the briefings, the explanations. What has changed is that the site no longer asks a writer to fill in a form.

Figures are from the replay described above and are informational, not investment advice.