Surrogate-Gradient Learning in Spiking Neural Networks

1. The problem & the trick

A spiking neuron's output is a hard threshold. Backprop needs a derivative everywhere. Something has to give.

Discrete-time leaky integrate-and-fire (LIF) neuron, soft reset:

$$U[t] = \text{decay}\cdot U[t-1] + I[t] - s[t-1]\,\vartheta \, ,\qquad \text{decay}=e^{-dt/\tau}\approx 0.9$$ $$s[t] = \Theta\big(U[t]-\vartheta\big)$$

$\Theta$ is the Heaviside step. Forward, that's exactly what fires. But its derivative is zero almost everywhere (and undefined at $U=\vartheta$):

$$\frac{d\Theta}{dx} = \begin{cases}0 & x\neq 0\\[2pt] \text{undefined ("}\infty\text{")} & x = 0\end{cases} \quad\Longrightarrow\quad \text{plain BPTT gives } \frac{\partial s}{\partial U} \equiv 0 \text{ a.e. — no learning signal.}$$

The surrogate-gradient fix: keep $\Theta$ in the forward pass, but in the backward pass substitute a smooth stand-in derivative — one of:

$$\textbf{SuperSpike}\text{ (Zenke \& Ganguli, 2018):}\quad \frac{\partial s}{\partial U}\bigg|_{\text{surrogate}} = \frac{1}{\big(1+\beta\,|U-\vartheta|\big)^{2}}$$ $$\textbf{sigmoid derivative:}\quad \frac{\partial s}{\partial U}\bigg|_{\text{surrogate}} = \beta\,\sigma\big(\beta(U-\vartheta)\big)\Big(1-\sigma\big(\beta(U-\vartheta)\big)\Big),\quad \sigma(x)=\frac{1}{1+e^{-x}}$$

sharpness β = 5.0

Forward pass — what the network actually computes

hard step Θ(V−ϑ) — forward smooth σ(βx) — the shape backward "pretends"

Backward pass — the gradient we actually use

true dΘ/dV (≈0, spike at V=ϑ) SuperSpike surrogate sigmoid-derivative surrogate

Drag β. Notice: the SuperSpike bump always peaks at height 1 but narrows; the sigmoid-derivative bump grows taller and narrower. Both converge toward the true (near-everywhere-zero, infinite-at-threshold) derivative as β→∞ — but neither is that derivative. That's the necessary fiction.

2. Train a tiny SNN in your browser

Task: two populations of input neurons (A = channels 0–3, B = channels 4–7) fire noisy, rate-coded spike trains at different rates each trial. The network must decide which population fired more spikes — trained end-to-end with backprop-through-time using exactly the surrogate gradient from Section 1.

Architecture: 8 inputs → 20 hidden LIF units → 2 leaky-integrator readouts, unrolled for 50 timesteps. Gradients are computed by hand for this exact unrolled graph (no autodiff library) — reverse-mode, timestep by timestep, using the surrogate derivative wherever the true spike derivative would appear. Trained online with Adam on freshly-sampled trials (an inexhaustible synthetic stream), so every click of Train sees new data.

learning rate = 0.0200

surrogate sharpness β = 5.0

surrogate function

iteration
0

loss (smoothed)
–

accuracy (smoothed)
–

Loss

Accuracy (dashed = chance)

Before training (frozen at reset)

–

Now (live, same input, current weights)

–

input spikes hidden LIF spikes

Pause & Think

The surrogate gradient is a fiction we inject into the backward pass — it does not exist anywhere in the forward computation, and nothing in the neuron itself computes it. Two questions worth sitting with:

What might play its role in the brain? Candidates on the table: feedback alignment (random, fixed feedback weights can substitute for the transpose of the forward weights); e-prop eligibility traces (a local, forward-running proxy for "how much did this synapse contribute recently," combined with a top-down error/neuromodulatory signal); and three-factor learning rules (pre-synaptic activity × post-synaptic activity × a global modulatory signal like dopamine, gating plasticity without a global backward pass).
Is BPTT itself biologically plausible? It requires storing the entire forward trajectory and running an exact, symmetric backward pass through time — i.e., the network must "know" its own history and weights precisely in reverse. Real synapses have none of the infrastructure for that. So even if you accept the surrogate gradient as a stand-in for the threshold nonlinearity, the deeper credit-assignment problem — attributing a delayed, global error signal to millions of individual synapses without literally reversing time — remains open.

3. 🎯 Challenges

Experiment with training to tick these off automatically.

Each item below is checked live against the actual network state — accuracy, loss, β, learning rate, and which surrogate you trained with. Nothing here can be ticked by hand; only training the network can do it. Progress latches: once ticked, an item stays ticked (saved in this browser) even after you reset the network.

test accuracy (held-out eval)
–

test loss (held-out eval)
–

0 / 8