我对 NetLogo 完全陌生,正在尝试创建一个基于代理的强化学习 (RL) 模型。我重新创建了一个玩具模型来获得帮助。
在这里,一个智能体通过与两个智能体交互来进行 RL,其中一个是不可靠的(因此与该智能体的交互没有得到加强),另一个是可靠的(因此与该智能体的交互得到了加强)。
但我不断收到错误:Extension exception: No urn specified
所以,很明显,我需要弄清楚如何指定骨灰盒!
项目文件夹可供您在以下位置签出:http ://bit.ly/1m8TAxq Bellow 是最小的、完整的验证代码,如下所示(尽管您需要上面链接的 dropbox 文件夹中的 URN 扩展来执行此操作) .
任何帮助将不胜感激。
extensions [ urn ] ;; the urn extension that contains the RL functions
breed [ agents agent ]
globals [ ;; Global variables
A1 ;; urn-ball name of agent 1 (RED)
A2 ;; urn-ball name of agent 2 (GREEN)
]
agents-own [ ;; Values assigned to each agent
main-urn ;; Name of the Polya urn used in RL
interactionPartner ;; Variable tracking who the learner is interacting with
knowledgeState ;; Variable tracking whether the learner knows the truth after each interaction
]
to setup ;; Setup the stage
clear-all
reset-ticks
setup-agents ;; Sets up the agents
end
to have-agents-setup-urns ;; Setting up agent polya urns for reinforcement learning (by observer)
ask agent 0 [
set main-urn urn:make-polya-urn initial-weight (list A1 A2)
;; There are two types of balls in the urn: A1, A2 corresponding to the two possible agents with whom the chooser may interact
]
end
to setup-agents ;; Setting up the agents in the population (by observer)
create-agents 3 [
setxy (-6 * cos ( who * 360 / population-size)) (12 * sin (who * 360 / population-size))
set shape "circle"
set size 4
]
ask agent 0 [ ;; This is the agent doing the reinforcement learning (BLUE)
set color [0 0 255 175]
]
ask agent 1 [ ;; This is the unreliable agent (RED)
set knowledgeState 0
set color [255 0 0 175]
]
ask agent 2 [ ;; This is the reliable agent (GREEN)
set knowledgeState 1
set color [0 255 0 175]
]
end
to step ;; The step function
ask turtles [do-game-play]
tick
end
to go ;; The go function simply calls step continuously
step
end
to do-game-play ;; The game play function (by observer)
ask agent 0 [
set interactionPartner urn:draw-ball main-urn ;; Draw from the agent urn to figure out with whom to interact.
if (interactionPartner = A1) [
set knowledgeState ( 0 ) ;; If the interactionPartner is agent 1 (who is unreliable), then set agent 0 knowledge state to 0.
]
if (interactionPartner = A2) [
set knowledgeState ( 1 ) ;; If the interactionPartner is agent 2 (who is reliable), then set agent 0 knowledge state to 1, and reinforce.
urn:reinforce main-urn A2 1
]
]
end