Quota follows execution context
BigQuery custom query quotas can constrain daily processed data for on-demand query workloads. A project-level limit covers the project's aggregate usage. A per-user limit applies separately to users and service accounts, but the project total still applies. If many applications submit queries as one service account, those applications share that identity's allowance.
This is different from setting a bespoke quota for one named caller. Official guidance states that the custom per-user setting is not individually assignable to a particular user or service account. The quota also does not represent every kind of BigQuery cost or a universal control for capacity-based pricing. Identify the pricing model and the particular quota before treating a failure as a spending emergency.
Prepare access to the failed job, regional job metadata, project quota settings, and the workload ownership map. Record the job's execution project, caller identity, location, and timestamp. The project containing the queried table may differ from the project where the job runs and consumes its relevant allowance.
Attribute the failure before increasing a limit
- Read the failed job with
bq show --job --format=prettyjson PROJECT_ID:LOCATION.JOB_ID, using the identifier from the actual failure. Confirm the quota error class and execution project. Distinguish daily processed-data quota from concurrent jobs, slots, API request rate, or a query-level bytes limit. - Inspect the project's custom query quota settings. Check both project and per-user limits, and compare the reset boundary with the job timestamp. Documentation uses a Pacific-time daily reset, so a local midnight assumption can produce a misleading incident timeline.
- Query the appropriate regional
INFORMATION_SCHEMA.JOBSview with a bounded time filter. Group relevant query jobs byuser_emailand workload labels where available. Account for the job metadata access required and avoid exposing unrelated query text in a broad incident report. - Coordinate a narrow response with the owners. Reschedule discretionary queries, optimize the offending workload, or approve a justified quota change. Separate service identities when workloads require distinct accountability, while recognizing that project-level usage remains shared.
Illustrative scenario
A fictional business-intelligence service and a nightly export use the same service account. The export scans unexpectedly broad data early in the day, after which ordinary dashboards fail under the per-user limit. The team identifies the common caller in job metadata and corrects the export. Raising every project's quota would obscure the attribution issue and increase exposure without improving the shared workload contract.
| Observation | Inspect | Appropriate response |
|---|---|---|
| One identity's jobs fail together | Per-user setting and caller | Review shared-account usage |
| Many identities fail in one project | Project aggregate allowance | Coordinate project-wide demand |
| Failure returns after local midnight | Documented reset boundary | Correct the timeline |
| Error concerns resources, not bytes | Exact job error and quota name | Follow the relevant capacity runbook |
Verify restored service
Run a small approved query under the affected runtime identity and execution project. Confirm the intended dashboard or scheduled workflow succeeds, then review new job metadata for correct attribution. Do not use a large production query merely to test quota availability. Check whether the source of excess demand has actually stopped before treating a temporary successful query as resolution.
Operating procedure
Maintain an identity-to-workload map and consistent job labels. Review shared service accounts when adding scheduled queries or new applications. Monitor quota failures, processed-data trends, and unexpected changes in caller mix. Document which control handles query bytes, total billing, and capacity so operators do not reach for the wrong limit during the next incident.
Explore Clouvenix cloud cost optimization for implementation and operational handover.
Technical reference
custom quotas, quota scope, and job metadata.