Upon much more inspection, I found the solution. It is an OS specific problem. When the file is deleted in Linux (or Macintosh) it just unlinks it. (I was not aware of this)
So if you run lsof on the machine, it still shows the file as open.
[user@machine]$ lsof | grep --color -i "testing.txt"
python26 26495 user 8w REG 8,33 23474 671920 /home/user/temp/testing.txt (deleted)
The solution is to stat the stream in python.
stat = os.fstat(fs.fileno())
Which will give you the number of links it has.
if stat.st_nlink < 1:
#has been deleted
And there you go. Now you know if you should reload it or not. Hopefully this helps someone else.