Context
A metric label that seems useful for debugging can create an unbounded number of time series. Request IDs, raw URLs, user identifiers, and order numbers grow with business activity rather than with a small stable set of service dimensions. The result can be expensive telemetry, increased SDK memory, and dashboards that become harder to query during an incident.
Consider an illustrative API that adds the complete request path to a duration metric. Paths containing account and document IDs now produce a new attribute combination for many requests. The service did not add a new feature, but its telemetry footprint grows rapidly. The fix begins by deciding which questions the metric is meant to answer.
Understand combinations rather than one label count
OpenTelemetry's Metrics SDK specification defines cardinality in terms of distinct attribute combinations and describes views, filtering, and cardinality-limit behaviour. Implementation support and configuration can vary by SDK version. OpenTelemetry also provides sensitive-data guidance, while Prometheus instrumentation guidance warns against unbounded dimensions. 123
A metric with several modest-looking labels can still create many combinations. Estimate the plausible combinations and compare them with actual observed series. Not every theoretical combination will occur, but the exercise reveals which dimension grows without a meaningful bound.
Start from an operational question
A service latency metric may need route template, method, outcome class, and service identity. It usually does not need a unique request ID to answer whether one route is slow. Individual request investigation belongs in appropriately controlled traces or logs with their own retention and privacy policy.
Write a short purpose statement for each important instrument. It helps reviewers reject attributes that add cost without improving the intended decision. A metric should be aggregatable in a way that preserves its meaning, rather than becoming a collection of one-off events disguised as time series.
Normalize routes and error categories
Use a stable route template such as an application route pattern rather than a raw path containing customer identifiers, where the instrumentation supports that contract. Group errors into bounded categories appropriate to the application. Do not place a full exception message or SQL statement into a metric attribute.
Keep normalization deterministic and test unknown routes. A fallback that records the raw unmatched path can recreate the original problem during scanning traffic or a new release. Decide how unsupported values are represented and monitor that category so lost detail is visible without becoming unbounded.
Apply filtering at the right point
Use supported SDK views or instrumentation configuration to retain only the attributes needed for the metric stream. Collector-side processing can help govern exported data, but filtering after the application already creates a huge local aggregation may not remove all SDK-side cost. Inspect the actual collection and aggregation path.
Check exporter and backend transformations as well. A resource attribute may be promoted into labels, or a dashboard query may combine dimensions differently from the raw export. The effective time-series population is an end-to-end property, so verify it at both the application and destination.
Treat identifiers as a privacy decision too
A hashed user ID can remain both high-cardinality and sensitive in context. Hashing does not automatically make a value anonymous or operationally useful. Avoid collecting personal identifiers unless the specific telemetry purpose and handling policy justify them.
Keep trace and log correlation through approved mechanisms with access controls and retention. Do not solve metric cost by moving the same unrestricted personal data into another telemetry stream without review. The goal is useful observability with an understood data contract, not merely a cheaper storage category.
Measure overflow and lost detail
If the SDK or backend enforces a cardinality limit, inspect how excess attribute sets are handled and whether an overflow aggregate is exposed. A hard limit can protect capacity while changing the detail available for analysis. Keep that behaviour visible so responders do not assume the remaining dimensions represent every request separately.
Do not choose an arbitrarily tiny limit just to make ingestion fit a budget. Test whether critical low-volume error categories remain observable during busy periods. An observability control should preserve the signals needed to detect failures, including failures that do not dominate the traffic mix.
Validate with representative and adversarial inputs
Generate a controlled range of route parameters, unknown paths, error messages, and tenant identifiers in a test environment. Observe SDK memory, exported series, backend ingestion, and query latency. The number of unique customer IDs should not create the same number of duration series if the intended route metric is bounded.
Repeat after instrumentation upgrades because defaults and semantic conventions can change. Keep the approved attribute set near the instrumentation code or configuration. A code review should be able to see when a new dimension changes the metric's cardinality or privacy assumptions.
Report savings without losing the purpose
Compare actual series and ingestion before and after, alongside the operational questions the revised metric can still answer. Preserve a migration plan for dashboards and alerts that used removed attributes. A lower series count is useful only when incident detection and investigation remain effective.
A focused operator exercise: A bounded-route instrumentation exercise
Generate requests to one route template with many different harmless resource identifiers, plus a controlled set of unknown paths. Observe the metric's exported attribute combinations and SDK resource use. Under a bounded route design, increasing unique identifiers should not create an equivalent unbounded set of latency series.
Repeat with an error message containing a changing request reference and confirm it is not promoted into a metric dimension. Check that responders can still move from an aggregate failure signal to authorized trace or log evidence. This validates both the cardinality budget and the usefulness of the remaining dimensions, rather than treating fewer series as the only objective.
Quick diagnostic map
| Evidence | Review | Acceptance boundary |
|---|---|---|
| Attribute sets | Inventory labels emitted by each instrument | Remove unbounded user and request identities |
| Budget boundary | Review limits and aggregation behaviour | Observe dropped or overflowed series |
| Operational value | Compare the resulting dashboard questions | Keep useful bounded dimensions |
Frequently asked questions
Can a request ID be useful in telemetry? Yes, in a suitable correlation context. It is usually a poor dimension for an aggregate metric.
Clouvenix can review instrumentation, SDK aggregation, and backend labels together, helping teams keep observability predictable as traffic and customer count grow.
For a scoped review of this workflow, discuss the relevant Clouvenix engineering service.