0

我以前得到:

self.memory = np.zeros((MEMORY_CAPACITY, s_dim * 2 + a_dim + 1), dtype=np.float32)  

但我需要在这个内存中添加一个变量“done”,所以我做了:

self.memory = np.zeros((MEMORY_CAPACITY, s_dim * 2 + a_dim + 2), dtype=np.float32)  

现在我在内存中添加了变量“完成”:

def store_transition(self, s, a, r, s_, done):
    transition = np.hstack((s, a, [r], s_, done))
    index = self.pointer % MEMORY_CAPACITY  # replace the old memory with new memory
    self.memory[index, :] = transition

所以现在它被添加了,但我还需要在我的其他函数中恢复它:

    indices = np.random.choice(MEMORY_CAPACITY, size=BATCH_SIZE)
    bt = self.memory[indices, :]
    bs = bt[:, :self.s_dim]
    ba = bt[:, self.s_dim: self.s_dim + self.a_dim]
    br = bt[:, -self.s_dim - 1: -self.s_dim]
    bs_ = bt[:, -self.s_dim:]
    bd = bt[:, here should be done]

所以 bd 应该包含 done 变量,我个人认为应该是:

 bd = bt[:, -1:] 

但我不确定....

此外,一些旧的位置必须改变,因为阵列变得更大,但我不知道哪个,什么以及如何....

任何可以帮助我的人吗?

4

1 回答 1

0

不太清楚你对这部分的意思还有一些旧的......

但是 numpy 切片语法有效。看这个例子:

>>> x = np.random.randn(5, 6)
>>> x.shape
(5, 6)
>>> x
array([[-0.66028509, -0.03515113,  0.54097151,  1.64021491,  1.55407344,
        -1.88961789],
       [-0.73310028, -0.38558638,  0.33200719, -0.142615  ,  0.57087033,
        -0.67726621],
       [ 0.32542737, -1.13508259,  1.58907859,  0.94438687,  0.33949198,
         1.52579515],
       [ 0.59211854,  0.39976888,  0.13617402,  0.57993582, -0.25274804,
        -1.15533191],
       [ 0.21203948,  0.72443024, -1.74406077,  0.97494208,  0.12653774,
        -0.00668887]])
>>> x[:, :-1]
array([[-0.66028509, -0.03515113,  0.54097151,  1.64021491,  1.55407344],
       [-0.73310028, -0.38558638,  0.33200719, -0.142615  ,  0.57087033],
       [ 0.32542737, -1.13508259,  1.58907859,  0.94438687,  0.33949198],
       [ 0.59211854,  0.39976888,  0.13617402,  0.57993582, -0.25274804],
       [ 0.21203948,  0.72443024, -1.74406077,  0.97494208,  0.12653774]])
>>> x[:, :-1].shape
(5, 5)
>>> x[:, -1:]
array([[-1.88961789],
       [-0.67726621],
       [ 1.52579515],
       [-1.15533191],
       [-0.00668887]])
>>> x[:, -1:].shape
(5, 1)
于 2019-02-20T15:01:43.040 回答