0

I'm implementing a monitoring system using GPS vehicle and my BIG PROBLEMNB is this, I have 100k data (GPS data) per second parallel to process.

Data is sent from many GPS information (GPSID, latitude, longitude, time), 100k data, with the same format for the second, this information should I process it (process it in some way (?)) To show the route of each gps in a visual system.

My big question is:

As I can "take" this large amount of data being sent every second? and after that process them in the most efficient way to display travel information for each GPS-id

Perhaps the process of grouping? for each GPS-id? I have not really clear things, any additional ideas or functionality will be helpful.

I would like to know some ideas of how to process this much information that applications use?, Algorithms? divided into several computers (how?)? ...

the most likely solution, involving architecture and algorithm, im working with java.

I've been reading about hadoop and map/reduce and i don not know if I could serve.

I just found it: https://github.com/nathanmarz/storm

any idea is appreciated

Info ad: GPS information is created by a script, I must see what the best way to receive such information and probably if I can skip a block of information, since 1 second of lost data blocks of coordinates think not affect the display and gives me more time to process the data, are possibilities, obviously the less information lost is much better

4

1 回答 1

0

一种简单的方法是不断接受来自客户端的数据并创建异步处理数据的请求。如果在处理开始时您知道结果将花费太长时间，则会跳过该项目并处理下一个项目。您可以考虑使用并发队列（下面的示例代码中未显示）来确保按照提交给执行程序服务的顺序处理项目。

    public static void main(String[] args) {
        final ExecutorService executorService = ...

        final long computationTime = 800; // say it tasks 800ms to process the data
        while (someCondition) {
           // receive gpsData from client
           executorService.submit(new ProcessingTask(computationTime, gpsData));
        }
    }

public class ProcessingTask implements Runnable {

    // want response inside a second
    private final long responseTime = System.currentTimeMillis() + 1000;
    private final long expectedComputationTime;
    private final Object gpsData;

    public ProcessingTask(long expectedComputationTime, Object gpsData) {
        this.expectedComputationTime = expectedComputationTime;
        this.gpsData = gpsData;
    }


    @Override
    public void run() {

        final long currentTime = System.currentTimeMillis();
        if (currentTime > responseTime + expectedComputationTime) {
            // the result will be available too late, skip processing this item
            return;
        }
        // TODO process gpsData
        // TODO send response to client
    }

}
于 2012-04-28T05:19:53.147 回答