javascript - Apify crawler with more than 2 clickable element

Question

I am trying to create an apify crawler, which has multiple clickable element. First click is to paginate, second click to visit each result, third is to visit a section of each result to extract more information.

function pageFunction(context) {
    var $ = context.jQuery;
    if (context.request.label === 'category'|| context.request.label === 'detail') {
        context.skipLinks();

        var result = {
            item_name: $('name').text(),
            categories: $('.categories').text(),
            email: $('email').text(),
            kvk: $('kvk').text()
        };
        return result;
    } else {
        context.skipOutput();
    }
}

The first 2 clicks are happening, it paginates and visits the results and extract first 3 values : item_name, categories and email

The fourth value : kvk is not returned. I think either the third click is not happening or the code I used have some errors. Can anyone please help me to fix this?

Answer 1

其中一个问题可能context.skipLinks()是阻止任何新的排队页面的功能。另外，您是否检查了开发者控制台中的所有选择器？对于调试，我建议您记录页面的内容，以便您知道它已加载。首先，您需要找到问题的根源。

附带说明，我建议您开始开发我们的现代网络抓取工具。爬虫平台不再维护，在某些情况下可能表现更差。