I don't have the Psychtoolbox, but this error message typically means that the (in this case) win
variable is not defined. Have you initialized this variable prior to calling the above lines of code?
The following link creating experiments using MATLAB and Psychtoolbox has some sample code and they define the win
variable as
win = Screen('OpenWindow',0, [900 900 1000], [10,10, 1100,1100]);
You will need to do something similar. Another link MATLAB cookbook does the following
% Initialize the screen with a black background
% rect is the coordinates of the screen
[win rect] = Screen('OpenWindow', 0, [0 0 0]);
ovalColor = [0 255 0]; % RGB color for the oval
rectColor = [255 0 0]; % RGB color for the rectangle
ovalRect = [100 100 300 200]; % Coordinates [x1 y1 x2 y2]
rectRect = [100 250 300 350]; % Coordinates [x1 y1 x2 y2]
Screen('FillOval', win, ovalColor, ovalRect);
Screen('FillRect', win, rectColor, rectRect);
Screen('Flip', win);
Try either option and see what happens.