reinforcement-learning - Q-learning：奖励计算的正确状态是什么

Question

Q学习-奖励

我正在努力解释 Q 学习算法的伪代码：

1  For each s, a initialize table entry Q(a, s) = 0
2  Observe current state s
3  Do forever:
4     Select an action a and execute it
5     Receive immediate reward r
6     Observe the new state s′ ← δ(a, s)
7     Update the table entry for Q(a, s) as follows:
8        Q( a, s ) ← R( s ) + γ * max Q( a′, s′ )
9     s ← s′

奖励应该从后续状态s'还是当前状态中收取s？

Answer 1

奖励应该从您执行操作后进入的后续状态a中收集。