2

I want to get a DistributionSummary over some domain data that does not change very frequently. So it is not about monitoring requests or sth like that.

Let's take number of seats in an office as example. The value for each office can change from time to time and there can be new offices and also offices get removed.

So now I need the current DistributionSummary over all offices, which needs to be calculated every time I think (similar to a Gauge).

I have a Spring Boot 2 app with micrometer and collect the metrics with prometheus and display them in grafana.

What I tried so far:

When I register a DistributionSummary, I can record all the values once during startup... this gives me the distribution, but calculated values like max get lost over time and I cannot update the DistributionSummary (recording new offices would work, but not changing existing ones)

// during startup
seatsInOffice = DistributionSummary.builder("office.seats")
    .publishPercentileHistogram()
    .sla(1, 5, 20, 50)
    .register(meterRegistry);

officeService.getAllOffices().forEach(p -> seatsInOffice.record(o.getNumberOfSeats()));

I also tried to use a @Scheduled task to remove and completely rebuild the DistributionSummary. This seems to work, but feels wrong somehow. Would that be a recommended approach? That would also probably need some synchronisation to not collect the metrics between removing and recalculating distribution.

@Scheduled(fixedRate = 5 * 60 * 1000)
public void recalculateMetrics() {
    if (seatsInOffice != null) {
       meterRegistry.remove(seatsInOffice);
    }
    seatsInOffice = DistributionSummary.builder("office.seats")
        .publishPercentileHistogram()
        .sla(1, 5, 20, 50)
        .register(meterRegistry);

    officeService.getAllOffices().forEach(p -> seatsInOffice.record(o.getNumberOfSeats()));
}

Another problem I just recognized with this approach: the /actuator/prometheus endpoint still returns the values for the old (removed) metrics, so everything is there mutiple times.

For sth like sla borders I could also use some gauges to provide the values (by calculating them myself), but that would not give me quantiles. Is it possible to create a new DistributionSummary without registering it and just provide the values it collected somehow?

meterRegistry.gauge("office.seats", Tags.of("le", "1"), officeService,
    x -> x.getAllOfficesWithLessThanXSeats(1).size());
meterRegistry.gauge("office.seats", Tags.of("le", "5"), officeService,
    x -> x.getAllOfficesWithLessThanXSeats(5).size());
meterRegistry.gauge("office.seats", Tags.of("le", "20"), officeService,
    x -> x.getAllOfficesWithLessThanXSeats(20).size());
meterRegistry.gauge("office.seats", Tags.of("le", "50"), officeService,
    x -> x.getAllOfficesWithLessThanXSeats(50).size());

I would like to have a DistributionSummary that takes a lambda or sth like that to get the values. But maybe these tools are not made for this usecase and I should use sth else. Can you recommend sth?

4

1 回答 1

0

DistributionSummary 有一个配置 distributionStatisticExpiry 可以控制旋转数据。这是一种解决方法

但是 PrometheusDistributionSummary 不会使用这个字段

           case Prometheus:
                    histogram = new TimeWindowFixedBoundaryHistogram(clock, DistributionStatisticConfig.builder()
                            .expiry(Duration.ofDays(1825)) // effectively never roll over
                            .bufferLength(1)
                            .build()
                            .merge(distributionStatisticConfig), true);
于 2021-07-22T07:49:09.623 回答