1

我正在努力提高我的应用程序中的搜索效率。我试图解决的用例是给定一组 Id，只在那些给定的行中执行搜索。问题是这个 id 的大小超过 1024。当我使用时，BooleanQuery.setMaxClauseCount(Ids.size()我没有得到 tooManyClauses 异常，但由于超时，它没有返回任何结果。

理想情况下，我想这样做：

public ResponseEntity search(
            @RequestParam(Param.SEARCH) String query,
            @RequestParam(value = Param.PAGE, defaultValue = Pagination.DEFAULT_PAGE) int page,
            @RequestParam(value = Param.SIZE, defaultValue = Pagination.DEFAULT_SIZE) int size)
            throws Exception {
        log.info("Started searching for query : " + query);

        final Set<Long> accessibleIds = projectsServiceUtils
                .filterIdsAccessibleByLoggedInPerson(null);

        final FullTextQuery fullTextQuery = searchHelper
                .getPrefixSearchQuery(Project.class, SearchFields.PROJECT_BOOST_MAP, query, accessibleRadarIds);

        final List<Project> projects = fullTextQuery.setFirstResult(page * size)
                .setMaxResults(size).getResultList();

        return ResponseEntity.ok()
                .header(Pagination.PAGINATION_PAGE, String.valueOf(page))
                .header(Pagination.PAGINATION_SIZE, String.valueOf(size))
                .header(Pagination.PAGINATION_COUNT, Long.toString(fullTextQuery.getResultSize()))
                .body(projectSearchObjectWriter.writeValueAsString(projects));
    }

以此作为我的 getPrefixSearchQuery 方法：

public <T> FullTextQuery getPrefixSearchQuery(
            Class<T> typeClass, Map<String, Float> boostMap, String searchTerms, Set<Long> ids) {
        FullTextEntityManager fullTextEntityManager = Search
                .getFullTextEntityManager(entityManager);
        QueryBuilder qb = fullTextEntityManager
                .getSearchFactory()
                .buildQueryBuilder()
                .forEntity(typeClass).get();

        BooleanQuery.Builder luceneQuery = new BooleanQuery.Builder();

        String[] tokens = searchTerms.split("\\s+");

        for (String token : tokens) {
            if (!StopAnalyzer.ENGLISH_STOP_WORDS_SET.contains(token) || tokens.length == 1) {
                // If search term contains only digits then search substring (possibly phone number)
                final String matcher = token.toLowerCase() + "*";

                final WildcardContext wildcardContext = qb.keyword().wildcard();
                TermMatchingContext termMatchingContext = null;
                for (String field : boostMap.keySet()) {
                    if (termMatchingContext != null) {
                        termMatchingContext = termMatchingContext.andField(field).boostedTo(boostMap.get(field));
                    } else {
                        termMatchingContext = wildcardContext.onField(field).boostedTo(boostMap.get(field));
                    }
                }
                final Query subQuery = termMatchingContext.matching(matcher).createQuery();
                luceneQuery.add(subQuery, BooleanClause.Occur.MUST);
            }
        }
        // NEW CODE TO SUPPORT FILTERING
        if (ids != null) {
            BooleanQuery.setMaxClauseCount(ids.size() + (tokens.length*boostMap.size()));
            TermMatchingContext termMatchingContext2 = qb.keyword().wildcard().onField("id");

            for (Long id : ids) {
                luceneQuery.add(termMatchingContext2.matching(id).createQuery(), BooleanClause.Occur.FILTER);
            }
        }

        FullTextQuery jpaQuery = fullTextEntityManager
                .createFullTextQuery(luceneQuery.build(), typeClass);

        return jpaQuery;
    }

由于使用上述配置我没有得到任何结果，我必须在获得查询结果后过滤我的结果。这会导致更多问题，因为我必须确保结果的大小等于传递的大小，因此我必须过滤整个结果集，然后对其进行分页以获得结果，而不是过滤结果的第一页所需的大小。这是我现在非常低效的工作：

public ResponseEntity search(
            @RequestParam(Param.SEARCH) String query,
            @RequestParam(value = Param.PAGE, defaultValue = Pagination.DEFAULT_PAGE) int page,
            @RequestParam(value = Param.SIZE, defaultValue = Pagination.DEFAULT_SIZE) int size,
            @RequestParam(value = Param.SORT, defaultValue = SortDefault.BY_NAME) String sort)
            throws IOException, IdmsException {
        log.info("Started searching for query : " + query);

        final FullTextQuery fullTextQuery = searchHelper
                .getPrefixSearchQuery(Project.class, SearchFields.PROJECT, query);
        final List<Project> projects = fullTextQuery.getResultList();

        final ImmutableSet<Long> Ids = projects
                .stream()
                .map(Project::getId)
                .collect(collectingAndThen(Collectors.toSet(),
                        ImmutableSet::copyOf));

        final Set<Long> accessibleIds = projectsServiceUtils
                .filterIdsAccessibleByLoggedInPerson(Ids);

        PageRequest pageRequest = new PageRequest(
                page, size);

        final Page<Project> projectsFiltered = projectRepository.findByIdIn(
                accessibleIds, pageRequest);

        return ResponseEntity.ok()
                .header(Pagination.PAGINATION_PAGE, String.valueOf(page))
                .header(Pagination.PAGINATION_SIZE, String.valueOf(size))
                .header(Pagination.PAGINATION_COUNT, String.valueOf(accessibleIds.size()))
                .body(projectSearchObjectWriter.writeValueAsString(projectsFiltered.getContent()));
    }

有什么方法可以在具有给定 ID 的行中执行搜索并对其进行分页？

4

1 回答 1

0 
    if (ids != null) {
       BooleanQuery.setMaxClauseCount(ids.size() + (tokens.length*boostMap.size()));
       TermMatchingContext termMatchingContext2 = qb.keyword().wildcard().onField("id");

       for (Long id : ids) {
           luceneQuery.add(termMatchingContext2.matching(id).createQuery(), BooleanClause.Occur.FILTER);
       }
   }

您正在使用通配符查询，但您不需要这样做。根据您的 ID 列表的大小，这可能是您超时的原因。

试试这个：

     if (ids != null) {
        BooleanQuery.setMaxClauseCount(ids.size() + (tokens.length*boostMap.size()));

        for (Long id : ids) {
            luceneQuery.add(qb.keyword().onField("id").matching(id).createQuery(), BooleanClause.Occur.FILTER);
        }
    }

此外，您不需要在每次执行查询时都设置 max 子句计数。实际上，您真的不应该这样做，因为可能会并行执行多个查询。

只需在应用程序启动时执行以下操作：

BooleanQuery.setMaxClauseCount(<some large number>);
于 2019-04-17T11:39:36.723 回答