2

This is a follow-up to my previous question.

Suppose I process my files in parallel. Now I would like to write the processing results to a file. Since the results do not fit in memory I cannot just wait until the processing of all files finish and then write the results. I have to do the processing and writing in parallel somehow.

For example: Suppose I have files with numbers. The file size is about 500M. The number of files is about 200. Each file fits in memory but all of them do not fit. Now I would like to write all even numbers found in these files to another file.

How to do that in Scala (with Futures, and Scala parallel collections)?

4

2 回答 2

5

在某些时候，您必须同步写作。如果您不想阻塞其他线程，一种可能性是使用参与者将结果写入文件。这可能看起来像这样：

class FileWriterActor(path: String) extends Actor {

  val file = ... // init FileWriter

  // this is how you implement an akka actor
  // plain scala actors look a bit different        
  def receive = {
    case x: MyResult => file.write(x.toString)
  }

  override def postStop() = file.close()
}

// usage
val result = ... // calculation stuff
fileWriter ! result
于 2012-07-21T06:17:58.480 回答
1

对于不熟悉akka的人：

import java.io.{File, PrintWriter}
import akka.actor.{Actor,ActorSystem,Props}

object AkkaWriterExample extends App{

  val outputPath : String = ???
  val system = ActorSystem("WriterSystem")
  val writer = system.actorOf(Props(new WriterActor(new File(outputPath))), name="writer")
  writer ! "this is a test"
  system.shutdown()
  system.awaitTermination()
}

class WriterActor(outFile: File) extends Actor {

  val writer = new PrintWriter(outFile)

  // this is how you implement an akka actor
  // plain scala actors look a bit different        
  def receive = {
    case str:String => println(str); writer.write(str);
  }

  override def postStop() = {
    writer.flush(); 
    writer.close();
  }
}
于 2015-08-18T15:04:53.633 回答