We have a list of the complete paths where we want to copy some files to, sort of like this
C:\temp\sub1\file1A
C:\temp\sub1\file1B
C:\temp\sub2\file2A
C:\temp\sub3\file3A
C:\temp\sub3\file3B
C:\temp\sub3\file3C
etc....
There is some code that does this
for each file in the file list
if (!Directory.Exists(dirName)
{
Directory.CreateDirectory(dirName)
}
And we can see from the logs that all the sub folders get created, and only 1 attempt is made for each sub folder - so the Directory.Exists is returning true after the first file in each sub folder has been through the above loop.
Then the code does something like
var fileDownloadActions = new List<Action>();
// populated the list with GetFile(requiredInfo)
Parallel.Invoke(new ParallelOptions {MaxDegreeOfParallelism = 10},
fileDownloadActions.ToArray());
And now here is the odd bit - most of the time this works just fine, however sometimes it fails with multiple DirectoryNotFoundExceptions - claiming it can't find the directories that have just been created!
Sometime it can't find any of the created sub folders, sometimes it works for some but not others.
Current working theory is that although the thread that created the folders thinks they are created (and the checks for Directory.Exists are returning true as no matter how many files we have in the folder we only try to create each folder once) they haven't been actually properly fully created on the OS and when we try to access them from the other threads it goes Bang!
Ever seen / heard of something like this before?
Is our working theory correct and if so what is the best way to fix this? (I'd rather not just put in a sleep!)