algorithm - Would non-decrease property change if the denominator is a non-invertible matrix in M step of EM algorithm?

Question

Suppose in the M-step of EM algorithm, the denominator of some parameters are matrix and they are non-invertible, we would use pseudo inverse matrix instead of it. If so, would the log likelihood still always increase?

I couldn't give a specific case and I fabricated this question. If you really need one, you could follow the EM algorithm of wiki page. In the filtering and smoothing part. suppose the denominator are matrix and the sum of them are not invertible, so what would happen for the loglikelihood? Still always increase?

Answer 1

对于任何特定情况，我建议您在该设置中完成 EM 算法的证明，例如https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm#Proof_of_correctness。一般来说，我希望只要您的 M 步增加值，它使整个 EM 通道最大化就会增加对数似然性，即使 M 步不是，例如，在每个处找到绝对最大值经过。

不过，如果这个不可逆矩阵意味着您已经进入了解决方案集的某个特殊区域，我仍然会担心。因为期望步骤计算出当前参数下的期望对数似然，一些特殊参数值，尤其是零，将意味着在最大化步骤中考虑的所有可能性共享这些特殊参数——有时，一旦一个参数变为零， EM 算法永远不会改变主意并将该参数从零移开。所以可能的情况是，一旦你得到一个不可逆矩阵，从那个位置开始的所有进一步的 EM 步骤也将具有不可逆矩阵，在这种情况下，你可能会发现 EM 算法很快就会陷入局部最优，之前它做了很多优化。