0

Part of a project I am working on requires me to read from a text file of unknown size and put the data into a 10KB buffer. I have to read the file into this buffer in 2KB chunks. I must have 4 threads running simultaneously. The first thread does the reading I just mentioned.

As the thread is reading in 2KB chunks of data into the buffer, the other three threads grab the 2KB chunk read in by the reader and conduct some calculations on it. Once these threads are done they grab the next 2KB chunk and do the same calculations. This is repeated until the entire text file has been read through.

Can anyone give me some guidance as to how to proceed with this part of the project? I know how to multithread. It is just the combination of multithreading and I/O that I have not had experience with before. I have only done simple I/O before.

4

1 回答 1

0

一种简单的机制可能是为 10K 缓冲区设置一个共享的“高水位线”。读取文件的线程（I/O 线程）将在适当的点更新它，“消费者”线程将读取它。它将受到互斥体的保护，当 I/O 线程更新它时，它会发出一个条件变量信号。只有 I/O 线程被允许写入该变量（同时持有互斥锁）和 10KB 缓冲区 - 其他线程被允许读取该变量（同时持有互斥锁）并且可以从缓冲区读取 - 但最多只能高水位线所指示的点。

一些非常粗略的伪代码（填写你自己的错误处理、检查完成、初始化等）：

pthread_mutex_t mux;
pthread_cond_var_t valid_bytes_updated;

int g_valid_bytes = 0;    // high water mark

char buffer[10 * 1024];

void* io_thread(...)
{
    int offset = 0;

    while (!done) {
        // read the next block of data
        readdata( file_handle, &buffer[offset], 2 * 1024);

        // let consumer threads know there's more data
        offset += 2 * 1024;
        pthread_mutex_lock( &mux);
        g_valid_bytes = offset;
        pthread_mutex_unlock( &mux);

        pthread_cond_broadcast( &updated);
}


void* consumer_thread(...)
{
    int processed_bytes = 0;

    while (!done) {
        // wait until there's something to do
        pthread_mutex_lock( &mux);
        while (processed_bytes == g_valid_bytes) {
            pthread_cond_wait( &valid_bytes_updated, &mux);
        }
        int valid_bytes = g_valid_bytes;
        pthread_mutex_unlock( &mux);

        // process the data in range &buffer[processed_bytes] to &buffer[valid_bytes], 
        //  keeping in mind that the range is up to but not including the byte at
        //  &buffer[valid_bytes]

        processed_bytes = valid_bytes;
    }
}
于 2012-04-24T23:51:02.733 回答