CIFAR-10 — DRAM-Native Binary Classifier

1. Experiment Data

• Code: github.com/aotto1968/forward-prop (otto-score-ifc/ tree)
• Dataset: forward-prop.nhi1.de/data/cifar-10-batches-bin/ (5 train + 1 test batch, binary planar RGB, 60000×3072 bytes) — fetch_cifar10.sh
• Project: forward-prop.nhi1.de

2. Leaderboard (2026-08-06)

Note: the 69.99% and 66.88% rows are ensemble-selection results (beam-search over stored per-member score archives), not single-config records. The 64.9% single-config record is the ceiling for one training run. See Status Report August 2026 for the 4-way comparison (maj1/maj3 × int32/flt32) and the precision findings.

2.1 Ensemble Search Breakthrough (2026-07-31 → 08-01)

The stored-score beam-search workflow breaks the 64.9% training ceiling: every channel×encoding×xform member is trained once and exported, then a beam search selects the optimal subset. Over 19,707 archives this reaches 69.99%:

Important (2026-08-06): the float32 score accumulator drifts on large sums (scores ~1e9 > 2²⁴) — the float merge found 29 members/66.79% where exact int64/double accumulation finds 44/67.72%. With exact 64-bit accumulation the merge is deterministic (--max beam-as-a-whole, --tries adds nothing) and the .ens archive follows the internal format (v12 double / v13 int64).

3. Architecture

Input:  3072 px → 11 encoding blocks × 256 uint32 = 2816 containers
Layer 0: W0 (frozen, random MAJ3): NC_slice × H
Layer 1: Bayes log-Score: H → K=10 (target + offset)
Voting:  Σ_members score[k]  →  argmax

Each W0 row picks 3 random containers, MAJ3 outputs 1 if ≥2 are set → frozen binary hash. The correction loop:

1. Score: score[k] = Σ_active target[k][bit] + offset[k]
2. Correct: If argmax ≠ true, target[true] += step, target[pred] −= step
3. Iterate N epochs with cosine step decay

4. Key Findings

4.1 Frozen W0 = fixed hash

Each neuron computes a fixed MAJ3 hash. If it fires for 57% of class A and 43% of non-A, the corrector can upweight it but cannot suppress the 43% false positives. This is the fundamental limit of a frozen representation.

4.2 More H gives √H returns — new record at H=1024

All runs: EN=3, ep=10, --multi-correct --splitVN 2. Only H varies.

Accuracy scales with √H: each doubling of H adds roughly +2-4pp up to H=1024. Beyond H=1024, returns invert — H=2048 reaches only 61.2% (−0.2pp) at 2× the compute. With spatial channels (ey-s-1+ey-s-2), H=512 reaches 63.5% at 16 epochs — the new sweet spot.

4.3 EN=3+ is the only lever — each EN has its own optimum epoch

Different W0 seeds draw from the same distribution. Ensemble voting with independent projections is the only lever: 3× independent random features beat 3× more features from the same projection.

All runs: H=512, encoding=ey-b,ey-a,ey-h,ey-s-1,ey-s-2 (17 members). EN and epochs vary.

Spatial channels need more epochs to converge (ep 16 vs ep 10 for color-only). splitVN=2 hurts spatial channels — they need full bit resolution.

From EN=3 onward, the ensemble–epoch correlation becomes visible. Each EN has its own optimum: EN=1 peaks at ep 8, EN=3 at ep 9, EN=7 at ep 10. More members suppress overfitting — the corrector can learn longer before eval stagnates.

Best choice: EN=3, ep=9 — 60.0% in 73s. EN=7 adds only +1.3pp for 2.6× the time.

4.4 splitVN — constant target bit-mass, only filter hardness changes

All V configurations have identical target bit-mass: 10 × H × 32/V × int32 = constant. When H is scaled inversely with V (e.g. H=256 at V=1, H=512 at V=2), the target matrix consumes the identical number of bits in memory. The only difference between V values is the filter hardness of the virtual neuron grouping.

All runs: EN=7, ep=10, --multi-correct. Only --splitVN and H vary. Target bit-mass is constant: H × 32/V = 8192 for all rows.

V=2 at H=512 is the champion — 61.3% in 187s. The AND2 filter removes 75% of noise while preserving enough signal. V=1 (soft majority) reaches 60.2% at the same bit-mass but lets more noise through. V=3 (AND3) and V=4 (AND4) filter too aggressively — the corrector starves even with more neurons.

4.5 Target init — three classes, one decisive metric

Seven init modes fall into three distinct classes. The decisive metric is per-neuron variation — whether different neurons have different target values for the same class.

Normal (intelligent) is consistently ∼2pp better than random. Whether count, dampen, inverse, or laplace — any data-driven co-occurrence structure gives the corrector a genuine head start. Polarity (count vs inverse) is irrelevant — the corrector converges to the same attractor regardless of sign.

Random (any seed) always correlates worse. Random per-neuron variation does enable learning (55.2% vs 10% for constant), but the lack of co-occurrence structure costs a consistent 2pp — independent of the seed. Variance between different seeds is <0.5pp.

Constant does not work at all. When all targets are identical per class (uniform: all = 1, prior: all = n_k), every class gets the same score for every sample. The corrector checks gap = sc[pred] − sc[true], finds gap=0, and fires zero corrections. Result: 10.0% (random chance). The corrector requires per-neuron leverage to differentiate classes.

4.6 Step schedule — cos-time always beats error-based control

Three step schedule families were tested at H=256, EN=1, ep=20, --multi-correct:

cos-time wins every time. The step follows a fixed cosine schedule independent of the current error. This gives the corrector the full learning budget — the step only decays at the very end, preventing late-stage oscillation without starving the early learning.

Pow (error-based) self-limits. pow=N computes step = step_init × (err/total)^N. As training error drops, the step shrinks — which limits further progress on both trn and evl. Higher N gives faster decay but lower peak accuracy. The best pow value (N=5) reaches 56.5%, still −0.7pp behind cos-time.

Pow-eval starves the corrector. Basing the step on eval error instead of training error sounded promising (auto-stop when eval plateaus), but eval error drops much faster than training error. The step collapses to near-zero by epoch 5-6, leaving 40-50% of the training budget unused. Even at very low power (0.5, giving step × √eval_err), the step stays high but oscillates — the corrector never stabilizes.

Conclusion: Time-based scheduling is the only safe choice. Error-based schedules (whether train or eval) limit the corrector prematurely. --gap-k (gap damping on step) can complement cos-time when overfitting needs to be suppressed.

4.7 Encoding repeat — diversity beats quantity

All runs: H=64, EN=1, ep=10, --multi-correct. Only the encoding repeat count varies.

ey-a, ey-b, ey-c are the same pattern permuted. All three use up+down+sig+sig on different raw color channels: ey-b → G, ey-a → B, ey-c → R. ey-a-2, ey-b-2, ey-c-2 are color-free variants that omit the raw color block (3 blocks instead of 4) — useful for testing whether color information drives performance, or whether opponent channels alone suffice. The performance alias combines ey-b-2,ey-a-2,ey-h,ey-s-2 for a 12-member color-free ensemble. After ey-a (B) and ey-b (G) are in the system, ey-c (R) adds no new encoding structure — only another projection onto the remaining raw color channel. The information from up+down+sig+sig on RGB is already captured by the first two combinations.

More members help — but only until the ceiling. Three different encodings (latest) reach 54.5% with 11 members, beating 5× repeat of any single encoding (53.0% max) which has only 4 members. Adding ey-c (15 members) improves to 55.1% — but the gain is from the extra members, not from encoding diversity. Repeating ey-b (15 members, 54.9%) gives almost the same improvement. At low H, any extra member helps.

At high capacity, more members saturate. Running the champion config (H=1024, EN=3, splitVN=2) with 4 encodings (ey-b,ey-a,ey-c,ey-h, 45 members) yields 61.2% — worse than latest (61.4%, 33 members). The extra 12 members add noise, not signal. Ensemble scaling does not work beyond ∼62% — the member ceiling has been reached.

Strong encodings converge at 4× repeat — ey-a and ey-b plateau at 53% after 4 repeats. Weaker encodings (ey-h, top-rgb) saturate earlier and lower. The 4-block encodings (ey-a, ey-b, ey-c) consistently outperform the 3-block ones (ey-h, top-rgb) by 3-5pp.

latest (11 encodings) is the empirically optimal combination.

5. Geometric Transform Ensemble (--xform)

Introduced 2026-07-16. Each active image transform (8 D4 transforms: id, hflip, vflip, dflip1, dflip2, rot90, rot180, rot270) becomes a new dimension in the member grid — like adding an extra --ensembleN but with structured geometric variation instead of random seeds. Four aliases are available:

• --xform all — all 8 D4 transforms
• --xform shift — all 12 pixel shifts (= augmentation - all)
• --xform performance — id, hflip, vflip, rot90 (4 transforms, faster experiments)
• --xform augmentation — all + shift = 20 transforms total

Note: CIFAR-10 results below use 5 transforms (flips + transposes); the 3 pure rotations were added later and not yet benchmarked on CIFAR.

Grid: ENSEMBLE[EN] × XFORM[XF] × COLOR[C] × HN[H] = EN × XF × C × H members
Each xform gets its OWN W0 + Target + Offset — trained independently.

5.1 Xform vs EnsembleN

Both multiply the member count, but through fundamentally different mechanisms. EnsembleN adds stochastic diversity (different random projections of the SAME data). Xform adds structured diversity (the SAME projection applied to transformed versions of the data).

All runs: --encoding latest (17 channels), 10 epochs.

Structured beats stochastic — xforms outperform ensembleN at every H. The gap widens with H: at H=1024, the five transforms can each express genuinely different features (+1.3pp).

5.2 Simple channels benefit most

Simple channel sets (ey-a = 4 members on B+opponents) gain massively (+8pp) because the transforms add genuinely new signal that the few channels cannot capture. The full latest encoding (17 channels covering color, opponent, spatial) already captures most of the geometric variation — xforms add only marginal diversity.

5.3 Architecture

1. ki_xform_raw() transforms raw uint8 pixel data before channel computation (Sobel, LBP, DoG, etc. see the transformed image → different texture signatures).
2. Each xform produces its own container buffer, independently cached via load_input_cached_xform().
3. Members pick their buffer via mem->input_buf. Member m with xform_id=x uses X_xform[x] for h0/gb precomputation, training, and evaluation.
4. No special eval path — the standard gb-cache mechanism works because each member's gb_buf was computed from its own xform's buffer.

5.4 Key Findings

1. Xforms are structural, not augmentation. Each xform gets independent W0, target, offset. Training targets are specific to each transform's view of the data.
2. All 8 transforms are pixel-verified — single-xform runs produce identical accuracy, proving the implementation correct.
3. Diagonal flips preserve spatial structure. dflip1 (transpose) and dflip2 (anti-diagonal) are valid geometric operations on square images — they shuffle pixels without destroying layout.
4. Gain scales with H. Small H limits per-xform capacity. H=1024 reaches +1.3pp over ensembleN (with 5 transforms).
5. 64.5% is the ceiling. Even with geometric diversity, the frozen MAJ3 projection cannot exceed this limit.
6. Xforms + encoding help low H, not the ceiling. More transforms or encoding members provide extra signal that matters most at low hiddenN — but they do not raise the ceiling. The ceiling is determined by H (hidden neurons), not by input diversity. The table below compares --xform all (8 D4) vs --xform performance (4 D4) at equal --encoding performance:

Interpretation: At H=8, doubling the transforms from 4→8 gives +1.7pp — the extra signal is meaningful because H is the bottleneck. At H=512, the same doubling gives only +0.7pp — H is no longer the bottleneck; the data ceiling dominates. More transforms or encoding members simply accelerate the convergence curve at each H — they cannot exceed the information capacity of H neurons.

Same pattern for encoding diversity: Section 4.7 shows that at H=64, more encoding members always help (54.5% with 11 members vs 53.0% with 5× repeat). But at H=1024 (Section 4.3), adding ey-c (15 members) reaches worse accuracy than latest (11 members) — the member ceiling has been reached. More members help only until H saturates.

5.5 Pure Rotations (rot90, rot180, rot270)

Added 2026-07-16. Three pure rotations complement the flip/transpose set, bringing the total to 8 transforms (--xform all):

rot180 preserves content while reversing all spatial gradients — especially promising for CIFAR where texture orientation matters (LBP, DoG, dir channels would see flipped patterns). Not yet benchmarked on CIFAR-10; the current 64.4% record uses 5 transforms (flips + transposes).

5.6 Pixel Shifts — Resolution Diversity

Added 2026-07-16. Twelve pixel shifts (up/down/left/right × 1/2/3 px) complement the D4 geometric set. Unlike D4 transforms (which permute pixels within the image), shifts move the MAJ3 projection window, exposing different 3×3 neighborhoods at each position. Vacated pixels are filled with 0 (black), making the shift visible to all channels (Sobel gradients, LBP texture, DoG edges all see the new border).

Key difference from D4: Shifts change the pixel content (some pixels lost to zero border), not just the arrangement. This creates resolution diversity — the corrector sees the same object at different sub-pixel alignments, forcing it to learn shift-invariant features. The --xform shift alias activates all 12 shifts at once. Combined with D4 transforms via --xform augmentation, the grid expands to 20 transforms × up to 17 channels = 340 members. Not yet benchmarked on CIFAR-10.

6. Ceiling Update: 64.9% Single-Config, 69.99% Ensemble Selection

Ceiling progression: encoding-only 61.4% → spatial channels 63.7% → geometric transforms 64.4% → --maj 1 --encoding-sizeN 32 64.9% (2026-07-24) → ensemble beam-search 69.99% (2026-08-01). Each technique adds orthogonal information sources that the frozen MAJ3 projection can exploit — but the single-training ceiling is ~65%; only the stored-score ensemble selection exceeds it.

Spatial channels (2026-07-14) break through the 62% encoding-only ceiling. Six spatial members (LBP, DoG, Variance, Gradient Direction, Local Range, Color-LBP on RG) combine with 11 color members to reach 63.7% (H=1024, EN=3, ep=16).

Geometric transforms (2026-07-16) add another +0.7pp via --xform all (tested with 5 transforms: hflip, vflip, dflip1, dflip2; now 8 with rotations). Each transform creates a new member dimension with independent W0, target, offset. Best: 64.4% (H=1024, EN=1, latest, 10ep).

Every accuracy improvement has come from a better channel or transform, not from an algorithm tweak. Target init (7 modes tested), step schedule (cos-time, pow, pow-eval), multi-correct, splitVN — none of these moved the needle beyond the encoding-only ceiling of 61.4%. Only adding genuinely new information sources (LBP → 62.3%, dir+range+lbp-rg → 63.7%, geometric transforms → 64.4%) produced measurable gains.

Critical nuance: input diversity helps low H, not the ceiling. More transforms or encoding members provide a genuine boost at small H (up to +2.1pp at H=64) — but the gain shrinks as H grows (+0.7pp at H=512). The ceiling of ~65% is determined by the number of hidden neurons, not by how many transforms or encoding members feed them. This is because the corrector has only H×32 bits of target memory — information capacity scales with H, not with input diversity. See Section 5.4 for the full data.

1. Frozen W0 — corrector can only reweight, not create features.
2. √H scaling — after H=512, diminishing returns dominate.
3. Geometric transforms: structured > stochastic — xforms add +0.4-1.3pp over ensembleN at equal member count.
4. Xform gain scales with H — at H=1024, each of 5 transforms expresses genuinely different features (+1.3pp).
5. Simple channels benefit most — ey-a (4 members) gains +8pp from 5 transforms. Latest (17 members) already captures most variation.
6. Training W0 collapses MAJ3 — destroys hash distribution.
7. Deeper layers lose information — 2-layer MAJ3 lost 4pp on MNIST.

6b. August 2026 Update — Precision, IFC v7, 4-way Comparison

Precision is the feature — the float32 accumulator drifts

Once the ensemble scores reach ~1e9 (far beyond float32's exact 2²⁴ range), the float32 score accumulator rounds on every addition and the beam's gain/argmax evaluations become noisy. The same corpus searched with different accumulators:

The "32-bit is free / only top bits matter" assumption is refuted. The DRAM chip accumulates popcounts exactly in integer arithmetic — the exact behavior is chip-faithful, the float32 drift was an emulation artifact. SCORE_TYPE defaults to double. The .ens archive follows the internal format (v12 double / v13 int64), and the .meta records/validates the arithmetic types. All format logic is centralized in lib/ki-ens.h (EnsReader + ens_write + ens_verify — the trainer verifies every export at write time).

The 4-way technology comparison (xform=id, 377 members)

Four corpora built identically (H=256, EP=10, seed 42, uniform W0), selection via --beam 10 --max (deterministic with exact accumulation):

• maj1 > maj3 consistently (+0.68-0.69pp cumulative, +1.11-1.44pp beam-selected).
• flt32-double ≈ int32 (±0.07pp cumulative) — with exact 64-bit accumulation the container format is neutral: the Container Principle holds once the drift is gone.
• Winner: M1-FLT32 = 66.88% (maj1 + float counters + double accumulation).

IFC v7 — the delivery tool handles heterogeneous ensembles

model.otto v6 could not represent a heterogeneous ensemble (the CIFAR import gave 12.81% = garbage). v7 adds per-member channel/encoding/ xform metadata; a second bug (missing enc-LUT init) gave exactly 10.0% = random. Both fixed — the IFC now runs the 51-member ensemble end-to-end: 66.36% == retrain == merge. --no-ens-cache provides real-live inference on new data (compute from raw, +814ms vs the file cache). Full details: Status Report August 2026.

7. Practical Recommendations

All configs use default count target init and cos-time step schedule. latest = 17 encoding members. --xform all = 8 D4 transforms (id, hflip, vflip, dflip1, dflip2, rot90, rot180, rot270), giving up to 136 total members. --xform augmentation = all 8 D4 + 12 pixel shifts = 20 transforms, giving up to 340 total members. The transform ensemble is a pure structural multiplier — no special eval path needed. CIFAR-10 results above used 5 transforms (the rotation set was added later). Shifts have not yet been benchmarked on CIFAR-10.