Status Report — July 2026

DRAM-native Classification via Bit-Logic Operations

​ 1. Target Initialization: Classical vs Random — Identical Convergence

Question: Does the initial value of the trained target/offset matrix affect final accuracy?

Experiment: Compare two init strategies for CIFAR-10 Otto Score (H=256, 5 ep):

Result: Both converge to the same final accuracy (~55%). Bayesian init gives a head start in epoch 1 (~30% vs ~10%), but by epoch 5 both are within 0.5pp. Pure zero initialization does NOT work — the correction loop needs a non-zero starting point to break symmetry.

Conclusion: The correction process is robust — target initialization does not determine the ceiling. The 55-57% barrier is architectural, not initialization-dependent.

​ 1b. Correction Distribution: Central vs Per-Sample — Same Ceiling

Question: Does distributing the log-odds correction across individual misclassifications (instead of one aggregated step) improve convergence?

Result: Both converge to the same final accuracy (~55%). The distributed approach shows slightly less oscillation during training (smoother accuracy curve), but at significantly higher computational cost. The ceiling is identical.

Conclusion: The correction architecture determines the ceiling, not how the correction is scheduled. The distributed mode's reduced oscillation does not translate to higher final accuracy — confirming that the 55-57% barrier is fundamental and not an optimization artifact.

​ 2. Encoding is Mandatory for Continuous Data — MNIST is the Exception

Finding: Binary neural networks (uint32 containers, XNOR+popcount operations) on continuous data (photographs: CIFAR-10) require a thermometer encoding of pixel values. MNIST is the exception: handwritten digits are binary by nature (ink/no-ink), so raw packing works directly.

MNIST — Special case: binary by nature

MNIST digits are inherently binary (>128 = ink, <128 = paper, slight anti-aliasing). Four binary pixels packed into one uint32 preserve the bit-level pattern — XNOR+popcount has structure to exploit. Encoding is optional for MNIST, not mandatory.

CIFAR-10 — Continuous data REQUIRES encoding

CIFAR-10 pixels are continuous 8-bit color values (0..255 per channel, 16M colors). Four adjacent pixels packed into uint32 have no bit-level correlation with visual similarity — XNOR between two random-looking containers yields random popcounts. Thermometer encoding restores the concept of “nearness” in bit space.

Why encoding matters: exp8 maps brightness to bit patterns where popcount(encode(pv)) ∝ pv. Pixel values 127 and 128 produce nearly identical bit patterns (differ by one bit) instead of completely unrelated ones. XNOR+popcount can exploit this structure — without it, the binary operations have no semantic gradient.

​ 3. Repository Structure

Current state (master branch):

otto-score-ifc/          ← Public distribution root
├── mnist/               ← MNIST trainers (Otto + Hebbian, unified)
├── cifar/               ← CIFAR trainers (symlinks → mnist/)
├── reference/           ← AdamW baselines
├── lib/                 ← Shared headers (maj3.h, ki-encoding.h, w0_random.h)
└── models/              ← Cached trained models

Only otto-score-ifc/ is part of the public distribution on GitHub. Local research directories (mnist-1/, cifar-1/, www/, etc.) are not in the remote repository.

Key architectural decisions

• Every trainer binary doubles as inference via --import — no separate IFC source files
• Encoding infrastructure in lib/ki-encoding.h — self-contained, no ki_Args dependency
• Hebbian uses the same multi-member architecture as Otto Score
• All 6 approaches are comparable at equal H and epochs

GitHub: github.com/aotto1968/forward-prop (master branch)

​ 4. The 60% CIFAR-10 Barrier — A Fundamental Limit

Claim: 60% accuracy on CIFAR-10 is a fundamental limit for DRAM-native bit-logic classifiers that use no dataset-specific prior knowledge.

Evidence

1. Single frozen random projection (Otto Score, Hebbian, AdamW all use this):
   • Best Otto Score result: 61.62% (July 2026, 17-seed ensemble, H=1024, VN=2)
   • Single-run ceiling: 61.0% (H=4094, VN=2, target-err=0.4)
   • Ensemble of independent seeds breaks the single-run ceiling: H=512 × 20 seeds → 61.22%, H=1024 × 17 seeds → 61.62%
2. Ensemble overcomes the single-run ceiling:
   • Single-run saturates at ~61.0% (H=4094)
   • Multiple seeds with different W0 streams give uncorrelated errors
   • EN=1..N merge-ensemble tool creates accuracy curves per seed
   • No hard ceiling yet — more seeds continue to improve accuracy
3. Target init invariance: Bayesian, random — all converge to same ceiling
4. Comparison with published results >60%:

Every published result >60% uses at least one of:

• Multiple passes over the data (backprop through W0)
• Data augmentation (random crops, flips, color jitter)
• Dataset-specific channel normalization
• Transfer learning from larger datasets (ImageNet)
• Batch normalization or similar adaptive scaling
• These techniques encode prior knowledge about natural images

Why 60% is the wall

• CIFAR-10 has 10 classes with 6000 images each
• A frozen random projection + MAJ3 + linear classifier has ~500K parameters
• The random projection destroys fine-grained spatial information
• MAJ3 further compresses 768 containers to 32 bits per neuron — severe information loss
• Without dataset-specific preprocessing, the model cannot distinguish classes that share coarse color/texture statistics

Position

We consider any CIFAR-10 result above 60% that does not disclose its full training pipeline (including data augmentation, transfer learning, and architecture search) to be potentially influenced by dataset-specific prior knowledge. The Kaggle CIFAR-10 leaderboard is not a valid comparison for DRAM-native classifiers because those entries use GPU-optimized deep learning with extensive prior knowledge built into the architecture.

​ 5. Decoupled Ensemble Workflow

Key architectural insight: Every Otto Score ensemble member is completely independent — different W0 (different random seed), trained from scratch, own target matrix. There is zero coupling between members. This enables a fundamentally decoupled workflow:

5.1 Train now, merge later

The --export-merge-scores DIR flag saves each member's per-class scores to an archive file (.ens) immediately after training. These archives contain everything needed to reconstruct the ensemble:

• All member scores (int64[n_test × n_classes])
• Ground truth labels (uint8[n_test])
• Configuration parameters (H, epochs, VN, seed, etc.)
• Per-member encoding metadata (color channel, encoding type/width)
• Embedded creation timestamp + ensemble eval accuracy

The merge-ensemble tool reads any number of archive files, accumulates scores in order, and produces an accuracy curve for EN=1..N. Training and scoring are completely decoupled:

┌────────────────────────┐     ┌─────────────────────┐
│  Train seed 1234       │ ──> │  scores/SD1234.ens  │ ─┐
│  --export-merge-scores │     └─────────────────────┘  │
└────────────────────────┘                              │
                                                        │   ┌──────────────────┐
┌────────────────────────┐     ┌─────────────────────┐  │   │  merge-ensemble  │
│  Train seed 5678       │ ──> │  scores/SD5678.ens  │ ─│─> │  scores/         │
│  --export-merge-scores │     └─────────────────────┘  │   │  → EN curve      │
└────────────────────────┘                              │   └──────────────────┘
                                                        │
┌────────────────────────┐     ┌─────────────────────┐  │
│  Train seed 9012       │ ──> │  scores/SD9012.ens  │ ─┘
│  --export-merge-scores │     └─────────────────────┘
└────────────────────────┘
   ... (N seeds, parallel, any time) ...

5.2 Maximum Parallelism

Because ensemble members are independent, the workflow enables:

• Parallel execution: N seeds can run simultaneously on N machines/cores — no shared state
• Incremental accumulation: Add more seeds days or weeks later — old archives are still valid
• Subset evaluation: --filter eval gt 58 excludes low-quality runs (text operators: gt, lt, ge, le, eq); --filter "GB=sig8" selects specific encoding channels
• Sorting flexibility: --sort seed or --sort ctime changes the EN curve interpretation
• Curve export: --save FILE produces plot-ready data for analyzing the EN accuracy saturation

5.3 Archive Versioning

File naming: H{hidden}_EP{epochs}_VN{splitVN}_HN{splitHN}_TE{te}_SD{seed}_F4_TS{timestamp}.ens

Archives from different training runs can be freely mixed as long as their configuration parameters (H, epochs, VN, etc.) match — the tool validates this and rejects mismatched files.

​ Summary

Andreas Otto — July 2026
github.com/aotto1968/forward-prop

Documentation Source
Markdown source: /docs/status-2026-07.html | All docs: /docs/ (synced from GitHub/docs)