1

I am working in OSX and using bash for my shell. I have a script which calls an executable hundreds of times, and each call is independent of the other. Therefore I am going to run this code in parallel. However, each call to the executable appends output to a community text file on a new line.

The ordering of the text file is not of importance (although it would be nice, but totally not worth over complicating since I can just use unix sort command), but what is, is that every call of the executable properly printed to the file. My concern is that if I run the script in parallel that the by some freak accident, two threads will check out the text file, print to it and then save different copies back to the original directory of the text file. Thus nullifying one of the writes to the file.

Does this actually happen, or is my understanding of printing to a file flawed? I don't fully know if this would also be a case by case bases so I will provide some mock code of what is being done in my program below.

Script:

#!/bin/sh
abs=$1
input=$(echo "$abs" | awk '{print 0.004 + 0.005*$1 }')
./program input

"./program":

~~Normal .c file stuff here~~
~~VALUE magically calculated here~~
~~run number is pulled out of input and assigned to index for sorting~~

FILE *fpp;
fpp = fopen("Doc.txt","a");
fprintf(fpp,"%d, %.3f\n", index, VALUE);
fclose(fpp);

~Closing events of program.c~~

Commands to run script in parallel in bash:

printf "%s\n" {0..199} | xargs -P 8 -n 1 ./program

Thanks for any help you guys can offer.

4

4 回答 4

3

在 open() 中设置附加标志的 write() 调用（如 fwrite()）（如在 fopen() 期间）可以保证避免您描述的竞争条件。

O_APPEND 如果设置，文件偏移量应设置为每次写入之前的文件末尾。

来自：开放的 POSIX 规范：

opengroup.org 打开

于 2013-06-05T02:33:14.827 回答
2

如果您可以选择，让您的程序写入标准输出而不是直接写入文件。然后你可以让 shell 合并你的程序的输出：

printf "%s\n" {0..199} | parallel -P 8 -n 1 ./program  > merged_output.txt
于 2013-06-05T13:11:53.177 回答
2

比赛条件是你所想的。

不是 100% 肯定，但如果你只是简单地附加到文件的末尾而不是打开它并编辑它应该是正确的

于 2013-06-05T01:36:36.923 回答
1

是的，这看起来像是灾难的秘诀。如果这些进程几乎同时打开文件，则只有一个会“接受”。

我建议（更容易）写入单独的文件，然后在处理完成后将它们组合在一起，或者（更难）将所有结果发送到将为每个人编写文件的消费者进程。

于 2013-06-05T01:38:59.343 回答