python - SGD optimiser graph

Question

I just wanted to ask a quick question. I understand that val_loss and train_loss is insufficient to tell if the model is overfitting. However, i wish to use it as a rough gauge by monitoring if the val_loss is increasing. As i use SGD optimiser, i seem to have 2 different trends based on the smoothing value. Which should i use? Blue is val_loss and Orange is train_loss.

From smoothing = 0.999, both seems to be decreasing but from smoothing = 0.927, val_loss seems to be increasing. Thank you for reading!

Also, when is a good time to decrease the learning rate? Is it directly before the model overfits?

Smoothing = 0.999

Smoothing = 0.927

Answer 1

根据我将深度学习应用于 CNN 的经验，过度拟合更多地与训练/验证精度/损失的差异有关，而不仅仅是其中之一。在您的图表中，很明显，随着时间的推移，损失的差异正在增加，这表明您的模型不能很好地泛化到数据集，因此显示出过度拟合的迹象。如果可能的话，它还可以帮助您跟踪 train 和 val 数据集的分类准确性——这将向您显示泛化错误，它充当类似的指标，但可能会显示出更明显的效果。

一旦损失开始平衡并且开始过度拟合，就降低学习率是个好主意；但是，如果您首先调整网络的复杂性以更好地拟合数据集，您可能会发现更好的泛化效果。对于这种过度拟合，适度降低复杂性可能会有所帮助——使用训练/验证损失和准确性的差异来确认。