Score → probability
The orange marker is the selected point: its score z becomes its probability p. The curve stays the same as the weights change; the point moves along it.
Train the classifier
Speed changes playback timing, not the learning rate.
Follow one prediction
Select any point in the scene. These calculations follow its inputs and the current weights.
Symbols and decision boundary
x₁ and x₂ are inputs; y is the true class, either 0 or 1. w₁ and w₂ are weights; w₀ is the bias. p is the predicted probability of class 1.
p = σ(z) = 1 / (1 + e−z)
The decision boundary is z = 0, where p = 0.5. Predict class 1 when p ≥ 0.5; otherwise predict class 0.
One step uses every point
- Compute each probability pᵢ using the current weights.
- Compare it with the true class: error = pᵢ − yᵢ.
- Average the contributions across all n points.
- Subtract learning rate × gradient from each parameter.
∂L/∂w₀ = (1/n) Σᵢ (pᵢ − yᵢ)
wⱼ ← wⱼ − η ∂L/∂wⱼ
η is the learning rate; j is input 1 or 2. All gradients use the same weights before the update.
What training minimizes
ℓᵢ = −yᵢ ln(pᵢ) − (1 − yᵢ) ln(1 − pᵢ)
Loss measures probability quality. Accuracy only counts which side of 0.5 each prediction falls on. Loss can improve while accuracy stays unchanged.
Train runs batch updates until the iteration limit. Pause stops the run; Step performs one batch update.
Three experiments
- Confidence without a new boundary.
Double both weights and the bias. The boundary stays in place, but probabilities move closer to 0 and 1. Inspect a point near the boundary.
- When classes overlap.
Choose Overlapping and train. Compare loss with accuracy: a confidently wrong prediction costs more than an uncertain one.
- A limit of a linear classifier.
Choose XOR or Moons. Training can move and rotate a straight boundary, but cannot bend it. A curved probability surface does not create a curved decision boundary.
Using the scene
Select a point to connect the geometry, sigmoid chart, and Math tab. Switch to 2D to add or delete points. Hollow points are misclassified.
Drag to rotate, Shift-drag to pan, and scroll to zoom. Side and Top-down keep their elevation fixed. Edit weights directly to explore a different starting classifier.
Training Metrics
| Points | 0 |
| Iteration | - |
| Log-Loss | - |
| Accuracy | - |
| Weights | - |
| Bias | - |
| Status | Add points to begin |
Latest batch update
Accuracy counts correct classes at p = 0.5. Log-loss measures the quality of the probabilities; lower is better.