CIFAR-10 — DRAM-Native Binary Classifier

1. Experiment Data

• Code: github.com/aotto1968/forward-prop (otto-score-ifc/ tree)
• Dataset: forward-prop.nhi1.de/data/cifar-10-batches-bin/ (5 train + 1 test batch, binary planar RGB, 60000×3072 bytes)
• Project: forward-prop.nhi1.de

2. Leaderboard (2026-07-14)

3. Architecture

Input:  3072 px → 11 encoding blocks × 256 uint32 = 2816 containers
Layer 0: W0 (frozen, random MAJ3): NC_slice × H
Layer 1: Bayes log-Score: H → K=10 (target + offset)
Voting:  Σ_members score[k]  →  argmax

Each W0 row picks 3 random containers, MAJ3 outputs 1 if ≥2 are set → frozen binary hash. The correction loop:

1. Score: score[k] = Σ_active target[k][bit] + offset[k]
2. Correct: If argmax ≠ true, target[true] += step, target[pred] −= step
3. Iterate N epochs with cosine step decay

4. Key Findings

4.1 Frozen W0 = fixed hash

Each neuron computes a fixed MAJ3 hash. If it fires for 57% of class A and 43% of non-A, the corrector can upweight it but cannot suppress the 43% false positives. This is the fundamental limit of a frozen representation.

4.2 More H gives √H returns — new record at H=1024

All runs: EN=3, ep=10, --multi-correct --splitVN 2. Only H varies.

Accuracy scales with √H: each doubling of H adds roughly +2-4pp up to H=1024. Beyond H=1024, returns invert — H=2048 reaches only 61.2% (−0.2pp) at 2× the compute. With spatial channels (ey-s-1+ey-s-2), H=512 reaches 63.5% at 16 epochs — the new sweet spot.

4.3 EN=3+ is the only lever — each EN has its own optimum epoch

Different W0 seeds draw from the same distribution. Ensemble voting with independent projections is the only lever: 3× independent random features beat 3× more features from the same projection.

All runs: H=512, encoding=ey-b,ey-a,ey-h,ey-s-1,ey-s-2 (17 members). EN and epochs vary.

Spatial channels need more epochs to converge (ep 16 vs ep 10 for color-only). splitVN=2 hurts spatial channels — they need full bit resolution.

From EN=3 onward, the ensemble–epoch correlation becomes visible. Each EN has its own optimum: EN=1 peaks at ep 8, EN=3 at ep 9, EN=7 at ep 10. More members suppress overfitting — the corrector can learn longer before eval stagnates.

Best choice: EN=3, ep=9 — 60.0% in 73s. EN=7 adds only +1.3pp for 2.6× the time.

4.4 splitVN — constant target bit-mass, only filter hardness changes

All V configurations have identical target bit-mass: 10 × H × 32/V × int32 = constant. When H is scaled inversely with V (e.g. H=256 at V=1, H=512 at V=2), the target matrix consumes the identical number of bits in memory. The only difference between V values is the filter hardness of the virtual neuron grouping.

All runs: EN=7, ep=10, --multi-correct. Only --splitVN and H vary. Target bit-mass is constant: H × 32/V = 8192 for all rows.

V=2 at H=512 is the champion — 61.3% in 187s. The AND2 filter removes 75% of noise while preserving enough signal. V=1 (soft majority) reaches 60.2% at the same bit-mass but lets more noise through. V=3 (AND3) and V=4 (AND4) filter too aggressively — the corrector starves even with more neurons.

4.5 Target init — three classes, one decisive metric

Seven init modes fall into three distinct classes. The decisive metric is per-neuron variation — whether different neurons have different target values for the same class.

Normal (intelligent) is consistently ∼2pp better than random. Whether count, dampen, inverse, or laplace — any data-driven co-occurrence structure gives the corrector a genuine head start. Polarity (count vs inverse) is irrelevant — the corrector converges to the same attractor regardless of sign.

Random (any seed) always correlates worse. Random per-neuron variation does enable learning (55.2% vs 10% for constant), but the lack of co-occurrence structure costs a consistent 2pp — independent of the seed. Variance between different seeds is <0.5pp.

Constant does not work at all. When all targets are identical per class (uniform: all = 1, prior: all = n_k), every class gets the same score for every sample. The corrector checks gap = sc[pred] − sc[true], finds gap=0, and fires zero corrections. Result: 10.0% (random chance). The corrector requires per-neuron leverage to differentiate classes.

4.6 Step schedule — cos-time always beats error-based control

Three step schedule families were tested at H=256, EN=1, ep=20, --multi-correct:

cos-time wins every time. The step follows a fixed cosine schedule independent of the current error. This gives the corrector the full learning budget — the step only decays at the very end, preventing late-stage oscillation without starving the early learning.

Pow (error-based) self-limits. pow=N computes step = step_init × (err/total)^N. As training error drops, the step shrinks — which limits further progress on both trn and evl. Higher N gives faster decay but lower peak accuracy. The best pow value (N=5) reaches 56.5%, still −0.7pp behind cos-time.

Pow-eval starves the corrector. Basing the step on eval error instead of training error sounded promising (auto-stop when eval plateaus), but eval error drops much faster than training error. The step collapses to near-zero by epoch 5-6, leaving 40-50% of the training budget unused. Even at very low power (0.5, giving step × √eval_err), the step stays high but oscillates — the corrector never stabilizes.

Conclusion: Time-based scheduling is the only safe choice. Error-based schedules (whether train or eval) limit the corrector prematurely. --gap-k (gap damping on step) can complement cos-time when overfitting needs to be suppressed.

4.7 Encoding repeat — diversity beats quantity

All runs: H=64, EN=1, ep=10, --multi-correct. Only the encoding repeat count varies.

ey-a, ey-b, ey-c are the same pattern permuted. All three use up+down+sig+sig on different raw color channels: ey-b → G, ey-a → B, ey-c → R. After ey-a (B) and ey-b (G) are in the system, ey-c (R) adds no new encoding structure — only another projection onto the remaining raw color channel. The information from up+down+sig+sig on RGB is already captured by the first two combinations.

More members help — but only until the ceiling. Three different encodings (latest) reach 54.5% with 11 members, beating 5× repeat of any single encoding (53.0% max) which has only 4 members. Adding ey-c (15 members) improves to 55.1% — but the gain is from the extra members, not from encoding diversity. Repeating ey-b (15 members, 54.9%) gives almost the same improvement. At low H, any extra member helps.

At high capacity, more members saturate. Running the champion config (H=1024, EN=3, splitVN=2) with 4 encodings (ey-b,ey-a,ey-c,ey-h, 45 members) yields 61.2% — worse than latest (61.4%, 33 members). The extra 12 members add noise, not signal. Ensemble scaling does not work beyond ∼62% — the member ceiling has been reached.

Strong encodings converge at 4× repeat — ey-a and ey-b plateau at 53% after 4 repeats. Weaker encodings (ey-h, top-rgb) saturate earlier and lower. The 4-block encodings (ey-a, ey-b, ey-c) consistently outperform the 3-block ones (ey-h, top-rgb) by 3-5pp.

latest (11 encodings) is the empirically optimal combination.

5. Ceiling Update: 63.7% via Spatial Channels

Spatial channels break through the 62% encoding-only ceiling. Six spatial members (LBP, DoG, Variance, Gradient Direction, Local Range, Color-LBP on RG) combine with 11 color members to reach 63.7% (H=1024, EN=3, ep=16).

Since record: 61.4% → 63.7% (+2.3pp) through spatial channel engineering.

Every accuracy improvement has come from a better channel, not from an algorithm tweak. Target init (7 modes tested), step schedule (cos-time, pow, pow-eval), multi-correct, splitVN — none of these moved the needle beyond the encoding-only ceiling of 61.4%. Only adding genuinely new information sources (LBP → 62.3%, then dir+range+lbp-rg → 63.7%) produced measurable gains. The color-only pixel evaluation seems exhausted at ∼62%. Future improvements will almost certainly come from new channel code that better connects pixels spatially — not from further algorithm optimization.

1. Frozen W0 — corrector can only reweight, not create features.
2. √H scaling — after H=512, diminishing returns dominate.
3. Seeds ≈ same distribution — EN is the only lever.
4. Spatial channels add orthogonal info — direction, range, chromatic texture provide features invisible to single-pixel channels.
5. splitVN=2 hurts spatial — AND2 filter discards too much signal from direction/range channels. Use no splitVN.
6. Training W0 collapses MAJ3 — destroys hash distribution.
7. Deeper layers lose information — 2-layer MAJ3 lost 4pp on MNIST.

6. Practical Recommendations

All configs use default count target init and cos-time step schedule. latest = 17 members (ey-b+ey-a+ey-h+ey-s-1+ey-s-2). Avoid --splitVN with spatial channels — they need full bit resolution.