pcp2openmetrics: add live timestamp mode#2637
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughSummary by CodeRabbit
Walkthrough
ChangesLive timestamp mode
Poem
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/pcp2openmetrics/pcp2openmetrics.1`:
- Around line 421-425: The SYNOPSIS short-option cluster is missing the newly
documented lowercase l option, so update the synopsis in pcp2openmetrics.1 to
match the documented command-line options. Keep the short-option list consistent
with the `-l, --time-stamp` entry by adding `l` in the compact options section
so users can discover it from the top-level usage summary.
In `@src/pcp2openmetrics/pcp2openmetrics.py`:
- Around line 142-146: The counter timestamp cache in pcp2openmetrics.py can
grow without bound because counter_timestamps only ever adds new (metric_name,
instance_id) keys. Update the live-mode counter handling paths around the
related metric processing code (including the sections near the referenced
counter-timestamp logic) to evict stale entries when instances disappear or are
no longer observed. Use the existing counter_timestamps structure as the cleanup
point so long-running exporter stability is preserved.
- Around line 58-59: The self.keys list in pcp2openmetrics.py has a missing
separator between the time_stamp and count_scale entries, causing those config
directives to merge into one invalid key. Update the list in the initialization
block where self.keys is defined so time_stamp and count_scale are separate
elements, preserving recognition of both directives.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: c37c1883-be10-4564-8af4-8c0dd5fd28c9
⛔ Files ignored due to path filters (1)
qa/1131.outis excluded by!**/*.out
📒 Files selected for processing (3)
qa/1131src/pcp2openmetrics/pcp2openmetrics.1src/pcp2openmetrics/pcp2openmetrics.py
| # Counter metric timestamp tracking | ||
| # key - (metric_name, instance_id) | ||
| # value - timestamp when first seen | ||
| self.counter_timestamps = {} | ||
|
|
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift
Prevent unbounded growth of counter_timestamps in long-running live mode.
self.counter_timestamps only adds keys and never removes stale (metric, inst) entries. With high instance churn, this can grow indefinitely and degrade long-lived exporter stability.
Also applies to: 509-513, 520-523
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/pcp2openmetrics/pcp2openmetrics.py` around lines 142 - 146, The counter
timestamp cache in pcp2openmetrics.py can grow without bound because
counter_timestamps only ever adds new (metric_name, instance_id) keys. Update
the live-mode counter handling paths around the related metric processing code
(including the sections near the referenced counter-timestamp logic) to evict
stale entries when instances disappear or are no longer observed. Use the
existing counter_timestamps structure as the cleanup point so long-running
exporter stability is preserved.
lmchilton
force-pushed
the
pcp2openmetrics-timestamps
branch
from
371f9c2 to
bcc4cc6
Compare
| if self.time_stamp and openmetrics_type(desc) == "counter": | ||
| counter_key = (metric, inst) | ||
| creation_ts = self.counter_timestamps.get(counter_key, ts) | ||
| body += '%s%s %s\n' % (openmetrics_name_ts, openmetrics_labels(inst, name, desc, labels), creation_ts) |
There was a problem hiding this comment.
I think its incorrect to expose timestamps for counter metrics only here - we should treat live mode the same as archive (a few lines above) in this regard, and export the timestamp for the sample (as per the v1 OM spec, and as we always have).
The "_created" lines are indeed intended for counters only, re-reading the spec(s). They could be applicable to both archive and live PCP values, however, looking at our code - and in particular the fetchgroup code behind the python API - these are not going to be handled correctly (there is no knowledge of "mark" records in src/libpcp/src/fetchgroup.c which is the code being used by pmconfig.py and in turn by pcp2openmetrics.py). We should ignore the _created timestamp, its not ever going to be correctly handled here.
Finally, the PMWEBAPI(3) uses the parameter name "times" to enable these timestamps - could we mirror that here? i.e. instead of --time_stamp lets use --times? (and this should remain the default in archive mode).
There was a problem hiding this comment.
@natoscott thank you for the clarification! I updated everything
lmchilton
force-pushed
the
pcp2openmetrics-timestamps
branch
from
bcc4cc6 to
3e0d882
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@qa/1131`:
- Around line 107-108: The new QA case uses pcp2openmetrics with only -s2, which
still falls back to the tool’s default 10-second interval and slows the test.
Update the command in this test case to pass an explicit short time interval
with pcp2openmetrics so the assertion remains fast while preserving coverage;
locate the change in the qa/1131 test around the pcp2openmetrics invocation.
In `@src/pcp2openmetrics/pcp2openmetrics.py`:
- Line 204: The `--times` option help text in `pmSetLongOption` is misleading
because the implementation in `pcp2openmetrics.py` only appends a sample
timestamp to emitted series rather than adding a new metric. Update the
`--times` description to reflect the actual behavior, and keep the wording
aligned with the logic used later when timestamps are emitted so users
understand what the flag does.
- Line 58: The config option handling in the pcp2openmetrics parser is wired to
the wrong runtime field, so `times=yes` never affects output. Update the option
mapping in the config parsing logic around the `times` entry so it sets
`self.time_stamp` (or rename the runtime flag consistently), and verify the
output gating that reads `self.time_stamp` now reflects the
`pcp2openmetrics.conf` setting.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: b8a86648-056d-4fd0-9d75-45cbdadeebdb
⛔ Files ignored due to path filters (1)
qa/1131.outis excluded by!**/*.out
📒 Files selected for processing (3)
qa/1131src/pcp2openmetrics/pcp2openmetrics.1src/pcp2openmetrics/pcp2openmetrics.py
| option may be used. | ||
| .TP | ||
| \fB\-l\fR, \fB\-\-times\fR | ||
| Live timestamp mode. When configured counters are accompanied by their respective timestamp |
lmchilton
force-pushed
the
pcp2openmetrics-timestamps
branch
from
3e0d882 to
8803d71
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/pcp2openmetrics/pcp2openmetrics.py (1)
507-510: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMinor duplication in timestamp-conditional formatting.
The two branches only differ by the trailing
%sforts; could be simplified to avoid repeating the format/label call.- if self.times: - body += '%s%s %s %s\n' % (openmetrics_name_end, openmetrics_labels(inst, name, desc, labels), value, ts) - else: - body += '%s%s %s\n' % (openmetrics_name_end, openmetrics_labels(inst, name, desc, labels), value) + line = '%s%s %s' % (openmetrics_name_end, openmetrics_labels(inst, name, desc, labels), value) + if self.times: + line += ' %s' % ts + body += line + '\n'🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/pcp2openmetrics/pcp2openmetrics.py` around lines 507 - 510, The timestamp-conditional formatting in the OpenMetrics output duplicates the same name/label/value formatting in both branches. Refactor the logic in the block around openmetrics_name_end and openmetrics_labels so the common prefix is built once, then append ts only when self.times is set. Keep the behavior identical while removing the repeated formatting call.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/pcp2openmetrics/pcp2openmetrics.py`:
- Around line 507-510: The timestamp-conditional formatting in the OpenMetrics
output duplicates the same name/label/value formatting in both branches.
Refactor the logic in the block around openmetrics_name_end and
openmetrics_labels so the common prefix is built once, then append ts only when
self.times is set. Keep the behavior identical while removing the repeated
formatting call.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 06360553-c000-4346-bdbb-cafe08f84372
⛔ Files ignored due to path filters (1)
qa/1131.outis excluded by!**/*.out
📒 Files selected for processing (3)
qa/1131src/pcp2openmetrics/pcp2openmetrics.1src/pcp2openmetrics/pcp2openmetrics.py
✅ Files skipped from review due to trivial changes (1)
- src/pcp2openmetrics/pcp2openmetrics.1
🚧 Files skipped from review as they are similar to previous changes (1)
- qa/1131
update pcp2openmetrics tool to allow for live timestamp mode option. This update follows openmetric / prometheus exposing timestamp guidelines. Updated man page & QA performancecopilot#1131 Resolves upstream RFE issue [performancecopilot#2538]
lmchilton
force-pushed
the
pcp2openmetrics-timestamps
branch
from
8803d71 to
2b4bd59
Compare
2 participants
update pcp2openmetrics tool to allow for live timestamp mode option. This update follows openmetric / prometheus exposing timestamp guidelines. Updated man page & QA #1131
Resolves issue #2538