Methods

Average layer-to-output Jacobian

A J lens uses fitted matrices to read an earlier residual-stream activation in the final layer's coordinate system. The model's final normalization and vocabulary projection convert the mapped activation into vocabulary scores and token ranks.

For source layer l, the fit averages the Jacobian from that layer’s residual stream to the target residual stream over prompt positions. The public lens contains one 2,048 × 2,048 matrix for each even source layer from 0 through 34. The target is layer 35.

J̄_l = mean_{prompt, position} ∂h₃₅ / ∂h_l

At readout time, the implementation maps the source residual with J̄_l h_l, applies the model's final normalization, and decodes the result with the unembedding matrix. The logit-lens baseline applies the final normalization and unembedding directly to the source residual.

Artifact configuration

Base model	WeiboAI/VibeThinker-3B
Model revision	77bd2cced09193c8b9a59a32bd8577bbd1f3e01c
Fit prompts	1,000
Source layers	0, 2, 4, …, 34
Target layer	35
Residual width	2,048

The fit used 1,000 WikiText prompts. The release does not include that fit corpus. The model repository records the artifact conversion and tensor checks. The original FP16 checkpoint used for the six browser traces is converted to Safetensors without changing its stored values.

Recorded readout result

The released evaluation input set contains 551 items across association, multihop, multilingual, order-of-operations, poetry, and typo suites. Each item pairs a prompt with one or more target terms. A SHA-256 split uses a 0.3 development threshold; the remaining items form the test split. The selected band was chosen on development items. The reported result uses 377 eligible test items.

The evaluator scores the final prompt token, except for poetry items, which use the last newline token. A target term is eligible when at least one case-and-spacing form maps to one tokenizer token. Order-of-operations items also use configured synonyms and number forms. Twelve input items had no eligible target term and produced no readout rows.

The evaluator completed readouts for 539 items, yielding 910 eligible target terms across both splits. Each target term has 18 J lens rows, 18 ordinary logit-lens rows, 18 shuffled-layer control rows, and one final-model row: 910 × 55 = 50,050. The token-target check adds no rows; each ledger row stores the selected foil token ID and foil rank alongside the target rank.

For each target term, the evaluator takes the best one-based vocabulary rank among eligible forms and then the best rank across the stated layer scope. Pass@k averages, across items, the fraction of each item's eligible targets with rank at most k. Aggregate MRR averages reciprocal best rank across eligible target terms. The paired-bootstrap MRR first averages within each item. The Final model row ranks the same targets in the model's next-token logits at the score position.

The selected band contains layers 24, 26, 28, 30, 32, and 34. For each target term, the token-target check selected one deterministic foil token from the global eligible-token pool after removing the item's eligible target IDs. The shuffled-layer mapping check replaced each source layer's lens matrix with the matrix nine entries ahead in the 18-layer list, with wraparound.

Each selected-band check used paired per-item differences between the J lens and each control for pass@10 and MRR, 2,000 resamples, and 95% percentile intervals. Both lower bounds had to exceed zero. The token-target and shuffled-layer mapping checks met that rule. The comparison with the ordinary logit lens required at least one lower bound above zero and did not meet that rule.

The overview table calculates each row independently over its named layer scope. Its all-layer aggregates are separate from the selected-band paired differences. Across all fitted layers, the shuffled-layer control matches the J lens on pass@10 and exceeds it on pass@1, pass@50, and MRR.

The released method file records the byte-level split algorithm, bootstrap random-number generator and seed, and configured order-of-operations synonym list used to recalculate the aggregate and control results.

The model package includes both lens files so each result remains paired with the tensor values used to produce it: FP32 for the evaluation and FP16 for the six traces. Casting every FP32 matrix to FP16 exactly reproduces the trace lens. The trace package contains the inputs, row ledger, aggregate, intervals, and method needed for recalculation.

The released evaluation contains ranked-token readouts only. It does not measure general model accuracy or evaluate causal steering, free-generation behavior, or global workspace behavior.

Captured prompt and continuation ranks

The dataset contains six synthetic scenarios. Each trace includes the rendered prompt, a model-generated continuation, tracked decoded-token rows, exact-position top-12 vocabulary ranks, and exported layers. Each continuation stops at the configured 64-token limit and usually ends before the answer is complete. The continuation may expose the model’s generated <think> text.

Browser scores are normalized ranks and relative indices. The release provides no calibrated probabilities or confidence estimates. The reference viewer preserves the self-contained layer × position pages used for direct inspection.

The pages display saved measurements. They do not run VibeThinker or calculate new J lens readouts. Analyzing a new prompt requires the companion source package, the lens artifact, the pinned base model, and local compute.

Version and checksum requirements

To reproduce a result, pin the base model, source, lens, and dataset revisions. The coordinated release manifest records these revisions and SHA-256 values. Local builds verify every copied trace file before creating the Pages directory.

Run python3 scripts/recalculate_readout.py in the trace repository to reproduce the selected band, aggregate metrics, paired controls, intervals, and decisions from the released rank rows. The script uses the Python standard library and does not run the model.