Virtual Study Hub

Introduce a small smoothing factor (epsilon) to avoid encountering expressions like 0 log 0, which cause underflow due to catastrophic cancellation. This tradeoff comes at the cost of slightly higher training time and model capacity, but ensures stable and accurate computation of cross-entropy losses.

Leave a Reply

Your email address will not be published. Required fields are marked *