Your Softmax Is Broken. Here's the Fix.
Jia-Bin Huang · 30:03 · Watch on YouTube
Standard cross-entropy in major frameworks can report a loss of exactly zero and a zero gradient on the correct class once a model’s logit margin crosses a floating-point precision cliff, even though the true loss is...