Network Diagram
Ready
Loss Curve
Controls
Step: 0
Backpropagation
Backpropagation is the algorithm used to train neural networks. It computes how much each weight contributed to the overall error, then adjusts weights in the direction that reduces the loss.
How to Use
- Forward Pass computes the output from inputs through the network
- Backward Pass computes gradients flowing from output back to inputs
- Train Step does both passes then updates weights using the gradients
- Adjust sliders to change inputs, target, learning rate
- Watch the log for step-by-step derivative computations
- Loss curve tracks the error over training steps
Forward Pass
- Input values
x1, x2enter the network - Each hidden neuron computes
z = w1*x1 + w2*x2 + b - Apply activation:
a = f(z) - Output neuron combines hidden activations
- Compute loss:
L = 0.5*(target - output)^2
Backward Pass
- Compute
dL/d(output) = output - target - Propagate gradient through output activation
- Compute gradients for output-layer weights
- Propagate gradient to hidden layer via chain rule
- Compute gradients for hidden-layer weights
Weight Update
- For each weight:
w -= lr * dL/dw - For each bias:
b -= lr * dL/db
Chain Rule
∂L/∂w = ∂L/∂a · ∂a/∂z · ∂z/∂w
The gradient of the loss with respect to any weight is the product of local gradients along the path from output to that weight.
Loss Function (MSE)
L = ½(t - y)²
Mean squared error between target t and prediction y.
∂L/∂y = y - t
Activation Derivatives
- Sigmoid:
σ'(z) = σ(z)(1 - σ(z)) - ReLU:
f'(z) = 1 if z > 0, else 0 - Tanh:
tanh'(z) = 1 - tanh(z)²
Weight Gradients
∂z/∂w = input to that weight
The gradient of a neuron's pre-activation with respect to a weight is simply the input that weight multiplies.
Metrics
| Step | 0 |
| Loss | - |
| Output | - |
| Target | 1.00 |
| Max |Grad| | - |
| Status | Ready |