0

似乎 GYM/stable_baselines3 与机器上的核心数量的扩展性很差。是“正常的”吗？还是我机器上的硬件配置问题？FPS 严重下降（随着时间的增加）。这是一些mwe

128 核：~100 FPS
$ time python3 mwe.py
real 2m35,323s
user 139m27,725s
sys 18m44,585s

2 核
$ taskset --cpu-list 1,2 python3 mwe.py
1300 FPS
真实 0m9,331s
用户 0m12,990s
sys 0m0,244s

32 核
$ time taskset --cpu-list 0-31 python3 mwe.py
800 FPS
真实 0m16,181s
用户 2m54,283s
sys 0m25,847s

from stable_baselines3 import PPO
import gym
from gym import error, spaces, utils
from gym.utils import seeding

#https://github.com/MatePocs/gym-basic/blob/main/gym_basic/envs/basic_env.py
class BasicEnv(gym.Env):
    metadata = {'render.modes': ['human']}

    def __init__(self):
        # There are two actions, first will get reward of 1, second reward of -1. 
        self.action_space = spaces.Discrete(5)
        self.observation_space = spaces.Discrete(2)

    def step(self, action):

        # if we took an action, we were in state 1
        state = 1
    
        if action == 2:
            reward = 1
        else:
            reward = -1
            
        # regardless of the action, game is done after a single step
        done = True

        info = {}

        return state, reward, done, info

    def reset(self):
        state = 0
        return state
  
    def render(self, mode='human'):
        pass

    def close(self):
        pass

env = BasicEnv()

ppo = PPO ("MlpPolicy", env,
        verbose=1, 
        learning_rate=0.0003
        )
ppo.learn(total_timesteps=10000)
4

0 回答 0