Skip to content

fix: add scrape timestamps to protobuf untyped metrics - #2718

Open
Tanishq-Choudhary wants to merge 1 commit into
prometheus:mainfrom
Tanishq-Choudhary:protobuf-untyped-timestamp
Open

Tanishq-Choudhary wants to merge 1 commit into
prometheus:mainfrom
Tanishq-Choudhary:protobuf-untyped-timestamp

Conversation

@Tanishq-Choudhary

Copy link
Copy Markdown
Contributor

The protobuf writer sets timestamp_ms from the scrape timestamp for every data point type except unknown. convert(UnknownDataPointSnapshot ...) in PrometheusProtobufWriterImpl never calls setScrapeTimestamp, so an unknown data point with a scrape timestamp loses it. The Prometheus text, OpenMetrics and OpenMetrics 2 writers all write it.

Prometheus reads timestamp_ms for untyped samples the same way as for counters and gauges (ProtobufParser.Series() in model/textparse/protobufparse.go). So untyped series scraped over protobuf get the scrape time instead of their own timestamp. One path that hits this is the simpleclient bridge: convertUnknown copies Sample.timestampMs onto the data point.

Repro on main:

UnknownSnapshot unknown =
    UnknownSnapshot.builder()
        .name("my_special_thing_bytes")
        .dataPoint(
            UnknownDataPointSnapshot.builder()
                .value(0.2)
                .scrapeTimestampMillis(1672850685829L)
                .build())
        .build();
new PrometheusProtobufWriterImpl().convert(unknown, EscapingScheme.ALLOW_UTF8);
// metric { untyped { value: 0.2 } } with no timestamp_ms

Fix: call setScrapeTimestamp in the unknown convert method like the other types do.

Test: testUnknownComplete in ExpositionFormatsTest now has a protobuf expectation with timestamp_ms, like testGaugeComplete. It fails on main and passes with the fix. testUnknownWithDots already checks that no timestamp_ms is written when there's no scrape timestamp, and still passes.

Ran unit tests for config, model, exposition-textformats and exposition-formats, plus google-java-format, checkstyle and the errorprone profile on the changed files.

The protobuf writer sets `timestamp_ms` from the scrape timestamp for
every data point type except unknown. An `UnknownDataPointSnapshot`
with a scrape timestamp is written without it, while the Prometheus
text and both OpenMetrics text writers include it.

Prometheus reads `timestamp_ms` for untyped samples the same way it
does for counters and gauges, so untyped series scraped over protobuf
lost their explicit timestamps. One path that sets them is the
simpleclient bridge, which copies `Sample.timestampMs` onto unknown
data points.

The writer now calls `setScrapeTimestamp` for unknown data points like
it does for all other types.

Signed-off-by: Tanishq Choudhary <tanishqchoudhary007@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant