This update step has the sign flipped: it adds the gradient instead of subtracting it, so the parameter walks uphill and the loss grows every step. This is the classic "my loss won't go down" bug. Fix line 3 to step against the gradient.
Expected output:
2.24
The break is on line 3 — but read the whole snippet first.