Elephant-1: a decision-model post-train through a general prompt census

I ran the elephant-1 model and its parent, Qwen3.5-9B, through the Cartographics HF space (a space that generates the model atlases referenced throughout this article, on demand). Base Labs at Baseten describes elephant-1 as a decision model: a full fine-tune of the language model on 6.21M decision examples, trained with cross-entropy plus a Brier calibration term and a reserved I-don't-know option, so that one shared-prefix prefill scores every option of a question set. This census asks what that post-training does to the general category organization the model inherited from Qwen3.5-9B.

Selective MLP entries fall from 13,889 to 11,054, a 20.45% reduction, and every one of the 17 labels loses entries. [data] Mean category-probe accuracy moves from 81.54% to 81.04%, a 0.50-point mean gap, lower at 29 of 32 layers with a largest layer-level gap of 1.42 points. [data] I tested whether decision training sharpens single-label selectivity for the usage labels; under the exported threshold it reduced selectivity for every label while leaving the high-F coordinate roster largely in place, at median within-layer F-rank correlation 0.953. [data]

Scope. This census measures text-input activations for 8,965 prompts across 17 usage labels. It does not measure images, the frozen vision encoder, the decision engine's calibrated readout, the serving-time block-bidirectional attention mask, or any generated output. Unless stated otherwise, results use the MLP intermediate stream at mean_all_real, averaging all real input tokens, template and special tokens included. The category probe is a five-fold, 256-PC multinomial classifier with training-fold PCA and scaling; the majority-class baseline is 6.58%. The parent atlas captures Qwen/Qwen3.5-9B at revision c202236235762e1c871ad0ccb60c8ee5ba337b9a; this atlas captures elephant-1 at revision b3f8a93462fb1051a9de031e94a7ff380e254072. The two runs share one corpus, one chat-template file and one tokenized rendering of it.

Evidence tags distinguish analytical claims: [data] is a direct measurement or calculation from the pinned atlas databases or capture metadata, [inferred] is my interpretation under stated assumptions, [data, inferred] is both, and [guess] is a mechanism I haven't tested.

Every label loses selective MLP entries, with roleplay and core_technical down a third

Under the exported threshold, Benjamini-Hochberg q < 0.01 with |AUROC - 0.5| >= 0.25, selective MLP entries move from 3.53% to 2.81% of the 393,216 MLP layer-coordinates: 2,111 rows gain the classification and 4,946 lose it. [data] The coordinates the tune dropped were not weaker than the ones it gained: median effect size is 0.2584 among lost entries and 0.2583 among gained. [data]

The proportional extremes are roleplay at -34.34% and core_technical at -33.15%, both falling faster than the population's 20.45% and losing 1.75 and 1.48 points of share. [data] The mildest is learning at -8.31%, which gains 0.78 points of share because its count decline is smaller than the population's; introspection gains 1.03 points of share the same way. [data] Losses concentrate late: 2,902 of the 4,946 lost entries sit in layers 22 to 31, and the band counts run 1,849 to 1,513, 3,865 to 2,863 and 8,175 to 6,678. [data]

Across all 12 captured streams, selective entries fall from 110,938 to 96,679 and mixed entries from 31,533 to 25,349 of the 1,720,320 feature rows. [data] Projected streams overlap computationally, so these are layer/component entries rather than a count of distinct physical units.

The developer's benchmarks put elephant-1 at 62.65 on the Jev Decision Index 0.4 and 65.2 on their Artificial Analysis replica; those are generated-output scores on decision tasks, and this census, which reads plain prompts at input positions, neither repeats nor contradicts them. [data]

Probe accuracy dips most at mid-depth

Mean MLP probe accuracy moves from 81.54% to 81.04%, lower at 29 of 32 layers; layers 0, 4 and 7 are the exceptions. [data] The band means run 80.08% to 79.76%, 81.64% to 81.00% and 83.04% to 82.48%, so the mid band takes the largest cut. [data] The largest single-layer drop is 1.42 points at layer 14; layers 12 to 17 fall between 0.75 and 1.42 points, while no late layer falls more than 0.84. [data]

Both atlases peak in the same neighborhood: the parent at MLP layer 8 with 84.67%, elephant-1 one layer earlier at 84.33%, where the parent reads 84.31%. [data] The parent's best accuracy-minus-control is 78.40 points at layer 8; elephant-1's best is 77.94 points at layer 21, where probe accuracy runs 84.23% to 84.09%. [data]

Nearest-neighbour label agreement at the last layer falls in every component: the MLP stream from 0.390 to 0.353 (3.64 points), gate from 0.444 to 0.424, v from 0.541 to 0.526, attn from 0.480 to 0.466, heads from 0.522 to 0.510, attn_gate from 0.506 to 0.495, up from 0.546 to 0.536, q from 0.461 to 0.453 and k from 0.417 to 0.414. [data] Both atlases peak at MLP layer 24, at 0.667 parent and 0.663 elephant-1. [data] Paired per-prompt probe errors are absent from the exports, so the 0.50-point mean gap carries no significance test.

The high-ranking roster persists while partnerships churn

Within-layer Spearman correlation of MLP-coordinate F ranks has median 0.953, with band medians 0.959, 0.950 and 0.954. [data] The curve dips to 0.931 at layer 15, recovers to 0.973 at layer 23 and ends at its floor, 0.904 at layer 31. [data] Median overlap of the top-256-F MLP sets is 79.69%, minimum 63.67% at layer 29. [data]

Coactivation partnerships break far more than the roster does: only 1,030 of 1,600 exported MLP pairs recur, 64.38% retention with Jaccard overlap 0.475, yet 1,360 parent pairs, 85.00%, keep both endpoints inside elephant-1's top-256 sets. [data] Across all streams, 8,406 of 10,800 pairs recur, 77.83%. [data] The high-F roster persists while the strongest selected partnerships change around it. [data, inferred]

Geometry expands in nearly every stream

Median per-component participation-ratio ratios sit above 1 in every band of every stream except two: the attention stream's late band at 0.997 and linattn_z's early band at 0.988. [data] The largest single-layer expansions sit deep in the full-attention blocks: attn_gate at layer 23 grows from PR 12.94 to 16.96 and at layer 27 from 11.28 to 14.07, and the late-band attn_gate median ratio is 1.248. [data]

The contractions cluster at the stack's entry: linattn_out at layer 1 falls to a 0.586 ratio and MLP layer 0 to 0.714. [data] The expansions and contractions move k90 together, so these are spectrum changes, not numerical drift. [data, inferred] The one expansion with a readout cost is linattn_out at layer 17, whose probe accuracy falls 2.80 points, from 72.68% to 69.90%, the largest drop of any stream at any layer. [data]

Elephant-1 adds one new geometry_dominated row, MLP layer 26 at last_input crossing the PR-below-3 flag; the parent's six dominated rows all recur. [data] The flag marks covariance concentration, not causal importance or dead activation.

The length-correlation profile of the leading principal components changes at several probed layers; correspondence between same-numbered PCs across the two captures has not been established. At the final MLP layer, PC1's length correlation moves from 0.8065 to 0.7729; at layer 7, PC3 moves from 0.0988 to 0.4788; at layer 8, PC2 from 0.0718 to 0.2325. [data] The summaries are not length-residualized, so prompt length remains a competing explanation for parts of the profile. [inferred]

At the prompt level, hub counts move the most: mean absolute hub-count change runs 1.66 to 2.90 across streams, with single-prompt changes up to 60 in the up stream, while the largest Mahalanobis z change is 1.09. [data]

Selectivity's peak moves deeper

The learned dictionary, taken at each stream's most selective layer, moved its chosen layer in 7 of 12 streams: mlp from layer 8 to 21, linattn_out from 8 to 24, v from 19 to 31, linattn_z from 29 to 26, gate from 25 to 26, linattn_qkv from 25 to 26 and up from 24 to 25. [data] The MLP move matches elephant-1's best accuracy-minus-control layer, 21. [data] The post-train pushed each stream's deepest single-label reading later in the stack. [data, inferred]

Among the 640 atoms whose stream kept its layer, 602 changed their top label at mean absolute AUROC changes of 0.040 to 0.057 per stream. [data] Dictionary-atom AUROCs sit near 0.5, so this is reassignment of weak signals rather than evidence of rewired circuits. [data, inferred]

Head-level structure mostly persists: 96 of 128 layer-head entries keep their leading norm-associated label. [data] The largest flip is head 11 at layer 15, learning to community, with norm ANOVA F falling 33.53 to 23.28; head 0 at layer 31 moves learning to creative_writing at F 65.43 to 56.77, and head 0 at layer 23 moves niche to tool_use while its F rises 22.09 to 29.53. [data] Mean head-norm F falls from 32.39 to 30.93. [data] The attn_gate view is the exception: 40 of 128 of its entries flip label and its mean F rises from 32.99 to 34.64. [data]

Inert feature rows rise from 347 to 396 of 1,720,320: 148 newly inert, 99 revived, 248 shared. [data] 74 of the new inert rows sit in MLP layers 0 to 2, matching the early-stack geometry contractions. [data, inferred] Health checks report no NaNs, infinities or dead coordinates in either atlas under the documented threshold. [data]

32 layers are mapped

Both atlases cover 24 linear-attention blocks and 8 full-attention blocks with identical recorded stream shapes and component coverage. [data]

The 216 component-layer pairs each carry last_input and mean_all_real summaries; the eight attn_gate streams add 16 sigmoid-position rows, 448 in total. [data] Stream widths: gate, up and mlp are 12,288 wide at every layer (393,216 rows each); attn, attn_gate, heads and q are 4,096 wide at the full-attention layers (32,768 rows each); k and v are 1,024 wide there (8,192 rows each); linattn_out and linattn_z are 4,096 wide at the linear-attention layers (98,304 rows each) and linattn_qkv 8,192 wide (196,608 rows). [data] The per-head table holds 128 heads, 128 q, 128 attn_gate, 32 k and 32 v rows, 448 in total. [data]

last_input is the newline, token id 198, ending the opened generation prompt, after the think block the template opened. Prompt length runs 12 to 60 tokens, median 26, with 10 template tokens of overhead and no truncation. [data]

Verdict: a fifth of the selective surface trades away while the readout holds

The result is 13,889 to 11,054 selective MLP entries with every label down, 81.54% to 81.04% mean probe accuracy with a largest layer-level gap of 1.42 points, median F-rank correlation 0.953, and 64.38% exported-pair retention against 85.00% endpoint survival. [data] The decision post-train cost about a fifth of the parent's single-label selective surface while keeping the high-F roster and the joint category readout largely intact: no layer's probe falls more than 1.42 points, and the deepest selectivity moves from layer 8 to 21. [data, inferred] The training's stated purpose, calibrated choices over labeled options, is invisible to this census by construction; what the census registers is a broad, mild erosion of the parent's category organization, deepest in the middle of the stack and re-centered deeper in every mapped stream. [data, inferred]

Limits

• The census exercises the standard causal text path only. The decision engine's calibrated readout, the serving-time block-bidirectional mask on full-attention layers, the frozen vision encoder and all generated output are not measured; matching stream shapes do not establish that the custom serving runtime was exercised. The developer's benchmark scores are quoted, not repeated.
• The two captures ran capture-code revisions 385f5e70 and dc0edf6. That diff changes portal admission, GPU and batching defaults, and Gemma4 MLP-width support; Qwen stream extraction, pooling, tokenization and runtime-dtype selection are identical in it, and the batch-token default resolves identically for a 9B capture. The corpus, chat-template and token fingerprints are identical across the two runs. Historical runtime precision and realized kernels remain unobserved.
• No paired per-prompt probe errors are exported, so probe gaps are reported as means and largest layer-level values without a significance test. Nothing here is an equivalence test.
• Usage labels are categories, not decision accuracy. The corpus contains no option sets or question criteria, so nothing in this atlas measures decision-model performance.
• Same-numbered principal components are not established corresponding axes across the two captures.

Atlases

Qwen3.5-9B · Clef-Flash · Qwen3.5-9B-Abliterated · more under juiceb0xc0de

All six tables are Parquet and in atlas/atlas.sqlite; the queries below use the local SQLite table names.

The selective counts by label; run this query against both pinned atlases for the paired counts:

SELECT top_label, COUNT(*) AS selective_entries
FROM features
WHERE component = 'mlp' AND taxonomy = 'selective'
GROUP BY top_label
ORDER BY selective_entries DESC, top_label;

The geometry and probe curves for the MLP stream, whose layer-14 row carries the pair's largest probe drop:

SELECT layer, geometry_participation_ratio, geometry_k90,
       geometry_twonn, geometry_dominated, probes_probe_accuracy,
       probes_probe_control_accuracy, probes_probe_selectivity,
       neighbors_knn_label_agreement
FROM layers
WHERE component = 'mlp' AND position = 'mean_all_real'
ORDER BY layer;

The roleplay coordinates, the steepest proportional loss:

SELECT layer, neuron, top_label, top_auroc, association,
       q, neg_log10_q
FROM features
WHERE component = 'mlp'
  AND taxonomy = 'selective'
  AND top_label = 'roleplay'
ORDER BY ABS(top_auroc - 0.5) DESC, neg_log10_q DESC,
         layer, neuron
LIMIT 20;

The head-norm label counts; the 96-of-128 retention claim additionally requires joining parent and elephant-1 rows on layer and head:

SELECT norm_top_label, COUNT(*) AS head_entries
FROM per_head
WHERE component = 'heads'
GROUP BY norm_top_label
ORDER BY head_entries DESC, norm_top_label;

Dataset revisions used for every number above: