I present SNN-Synthesis v9, a comprehensive investigation of stochastic resonance in neural networks spanning 63K-parameter CNNs to 7B-parameter LLMs across 67 experimental phases. Building upon v1–v8 (Phases 1–63), this version adds 4 new experiments (Phases 64–67) that establish four new findings. Four New Findings (v9) (XVI) Cross-Task SR-Quantization — 4-bit Qwen-1.5B achieves 58% at K=1 on arithmetic, surpassing FP16 (32%) by +26pp. At K=11, 4-bit (84%) > FP16 (76%). The space-time-precision triad generalizes beyond Modified Hanoi. However, on GSM8K (floor: 2%) and Factual QA (ceiling: 98%), no effect is observed—SR requires intermediate baseline competence. (Phase 64) (XVII) Noise Source Separation: Hook vs. Temperature — Decoupling weight-space noise from sampling noise reveals: hook noise alone (90.0%) > temperature alone (86.7%) > both combined (83.3%) > greedy (63.3%). Combining both is worse than either alone—destructive interference between noise sources. Temperature-diverse beam search provides +23pp over greedy without any model access, enabling API-only stochastic resonance. (Phase 67) (XVIII) The Irreversibility of Spatial Destruction — Weight pruning at 50% causes near-total collapse (K=1: 0%, K=21: 3.3%). At 70–90%, even K=21 NBS yields 0%. Unlike quantization (which perturbs weights → SR), pruning deletes information → irreversible destruction. Perturbation enhances; deletion destroys. (Phase 65) (XIX) Ensemble Ratio Law — Sweeping Mistral:Qwen ratio from 11:0 to 0:11 reveals accuracy saturates at ≥6:5 Qwen allocation (90.0%). The architectural diversity premium is task-dependent: on arithmetic where Qwen dominates, adding Mistral dilutes rather than diversifies. (Phase 66) v1–v8 Foundations (Phases 1–63) (I) Noisy Beam Search: K=11 parallel noisy trajectories achieve 78% on CNN (from 12%) and 100% on Mistral-7B Modified Hanoi (from 16%).(II) SNN-ExIt: Oracle-free self-evolution reaches 99% on ARC-AGI-3 LS20, surpassing Oracle CNN (78%).(III) Knowledge Multiplexing via ID Gating (Phase 35c): A single 115K parameter CNN stores distinct knowledge for multiple games without interference, using discrete condition-ID gating. Knowledge separation score reaches +0.572.(IV) σ-Diverse NBS (Phase 37a): Assigning different σ values to each of K=11 beams eliminates the need for task-specific σ* tuning.(V) Capacity Scaling (Phase 38a): ID gating requires ≥2.7K parameters for effective knowledge separation.(VI) Multi-Model NBS (Phase 38): Qwen2.5-7B-Instruct achieves 100% solve rate at K=11, confirming cross-model universality.(VII) GSM8K LLM-ExIt (Phase 33): Extending LLM-ExIt to math reasoning.(VIII) The Bitter Lesson (Phases 44–46): Overhead >0.5ms is fatal under time constraints.(IX) SimHash O(1) Curiosity (Phase 51): Matching RND at ~100× less overhead.(X) SR-Quantization (Phase 59): Qwen-1.5B + NBS (80%) > Mistral-7B baseline (42%).(XI) TTC Scaling Law (Phase 60): Logarithmic accuracy scaling with K.(XII) Quantization Noise as SR (Phase 61): 4-bit at K=1 (58%) > FP16 (32%) > 7B greedy (42%).(XIII) Multi-Model Beam Ensemble (Phase 63): Mistral+Qwen mix achieves 86.7%.(XIV) ARC-AGI-3 Kaggle Field Validation: Simplest agent (v5, 0.13) beats all "intelligent" agents.(XV) 6 Null Results (Phases 53–55, 57–58): Confirming design convergence. 42 contributions spanning 63K–7B parameters, CNNs to Transformers, 9 task domains, 2 model families (Mistral, Qwen), 4 model scales (1B–7B), and 22 honest null results.Code and data: https://github.com/hafufu-stack/SNN-Synthesis Acknowledgments This research was conducted entirely independently, without institutional affiliation or corporate funding. The author currently faces financial constraints that make it increasingly difficult to maintain subscriptions to AI services essential for this line of research. To sustain and improve the quality of future work, the author is actively seeking community sponsorship. Details are available at https://github.com/sponsors/hafufu-stack.
Building similarity graph...
Analyzing shared references across papers
Loading...
Hiroto Funasaki
Building similarity graph...
Analyzing shared references across papers
Loading...
Hiroto Funasaki (Tue,) studied this question.
www.synapsesocial.com/papers/69e07de52f7e8953b7cbede5 — DOI: https://doi.org/10.5281/zenodo.19562871