Cloud Logging In Depth: Buckets, Views and Queries

Everyone can find a log in the console. Far fewer can say where it is stored, how long it stays, who can read it, and what it costs — and those four questions are the whole of running logging at scale.

Topic 1: Where a Log Entry Actually Lives

Two buckets exist in every project without you creating them:

_Required is why “we deleted the audit trail” is not a thing that happens on GCP. It also means an attacker with project-owner cannot shorten it.

Custom buckets are what you create for anything that needs different retention or different access:

gcloud logging buckets create security-logs \
  --location=europe-west1 --retention-days=1095 \
  --description="Data Access audit logs, 3-year retention"

# Retention on _Default, if 30 days is not enough
gcloud logging buckets update _Default --location=global --retention-days=90

Location matters and is immutable. A bucket created in global cannot be moved to europe-west1 later, and for a data-residency requirement that is the setting that carries it.

Topic 2: Sinks Put Entries in Buckets

A sink with a bucket destination is how logs reach a custom bucket. The routing lesson covers sinks in full; the part specific to buckets:

gcloud logging sinks create security-to-bucket \
  logging.googleapis.com/projects/acme/locations/europe-west1/buckets/security-logs \
  --log-filter='logName:"cloudaudit.googleapis.com%2Fdata_access"'

An entry can land in more than one bucket. Sinks are not exclusive: the same entry can go to _Default, a custom bucket, and BigQuery, and you pay ingestion for each. That is the most common source of a surprising logging bill.

Exclusions are how you stop paying for noise, and they apply per sink:

gcloud logging sinks update _Default \
  --add-exclusion=name=health-checks,filter='
    resource.type="http_load_balancer"
    httpRequest.requestUrl=~"/healthz$"'

Health checks, readiness probes and load balancer polling are usually the single largest category of logs in a Kubernetes project, and they carry almost no diagnostic value. Excluding them commonly cuts ingestion by a third or more — measure before and after rather than guessing.

Exclusion is not deletion. An excluded entry is never ingested, so it is not searchable anywhere afterwards. Exclude things you are certain you will never need, and prefer routing to a cheap bucket over excluding when you are unsure.

Topic 3: The Query Language, Properly

Most people use a substring search and stop. The filter language is small and repays half an hour:

# Scope: always start here, it is the biggest speed win
resource.type="k8s_container"
resource.labels.cluster_name="prod"
resource.labels.namespace_name="checkout"

# Severity is a comparison, not a string match
severity>=ERROR

# Field access, with the JSON payload
jsonPayload.user_id="u_8812"
jsonPayload.duration_ms>1000

# Substring, regex, and existence
textPayload:"connection refused"
jsonPayload.message=~"timeout after \d+ms"
jsonPayload.stack_trace:*

# Boolean, with explicit parentheses
severity>=WARNING AND (jsonPayload.tenant="acme" OR jsonPayload.tenant="globex")

# Time — relative works and is what you want in a saved query
timestamp>="2026-08-19T10:00:00Z"

Three operators worth distinguishing, because mixing them up is why a query returns nothing:

• = is exact. textPayload="error" matches only the entry whose entire payload is error.
• : is “contains”. textPayload:"error" is what you almost always meant.
• =~ is a regular expression, RE2, and slower — scope the query first.

Always constrain resource.type and a time range. A query across every log type in a busy project is slow and sometimes times out; the same query scoped to one resource type returns instantly.

gcloud logging read '
  resource.type="cloud_run_revision"
  resource.labels.service_name="checkout"
  severity>=ERROR' --limit=50 --freshness=1h --format=json

Topic 4: Log Analytics — SQL Over Your Logs

Upgrading a bucket to Log Analytics gives you SQL over its contents at no extra storage cost:

gcloud logging buckets update _Default --location=global --enable-analytics

SELECT
  TIMESTAMP_TRUNC(timestamp, HOUR) AS hour,
  JSON_VALUE(labels.service) AS service,
  COUNTIF(severity = 'ERROR') AS errors,
  COUNT(*) AS total,
  SAFE_DIVIDE(COUNTIF(severity = 'ERROR'), COUNT(*)) AS error_rate
FROM `acme.global._Default._AllLogs`
WHERE timestamp > TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 24 HOUR)
GROUP BY hour, service
ORDER BY hour DESC;

This is what the filter language cannot do: aggregation, joins, percentiles, and comparisons across time windows. “Which endpoint’s p99 latency regressed after Tuesday’s deploy” is a SQL question, not a search question.

Enabling analytics is not the same as exporting to BigQuery. Analytics queries the log bucket in place — no duplicate storage, no second ingestion charge, and retention still governed by the bucket. Exporting to BigQuery is a separate choice, for when you need to join logs against business tables or keep data beyond the bucket’s retention.

_AllLogs is the view of everything in the bucket; a linked BigQuery dataset lets you query it from BigQuery itself, which is how you join it to anything else.

Topic 5: Log Views — the Access Boundary

Buckets hold entries; views control who can read which subset. This is the mechanism for “the support team can see application logs but not audit logs”.

gcloud logging views create app-only \
  --bucket=_Default --location=global \
  --log-filter='resource.type="cloud_run_revision" OR resource.type="k8s_container"'

gcloud logging views add-iam-policy-binding app-only \
  --bucket=_Default --location=global \
  --member=group:support@acme.example --role=roles/logging.viewAccessor

The grant that undoes it: roles/logging.viewer or roles/logging.privateLogViewer at the project level bypasses view restrictions entirely. If you are using views as a boundary, audit for project-level logging roles — otherwise the view is decorative.

privateLogViewer is the one to treat as privileged. Data Access audit logs and anything classed as private require it, and it should be a short list of people with a reason.

Topic 6: Cost, and Keeping It Honest

The billing model, in the order the money goes:

ingestion    ~$0.50/GiB beyond the free 50 GiB/project/month   ← where the bill is
storage      free for the bucket's default retention; charged beyond it
_Required    always free
analytics    free to enable; queries on a linked BigQuery dataset are billed

Ingestion dominates. Retention is comparatively cheap, which means the lever is what you ingest, not how long you keep it.

The routine that keeps it under control:

□ find the top log producers monthly
□ exclude health checks and probe traffic from _Default
□ turn down application log levels in production — DEBUG in prod is a bill
□ check whether the same entry is being routed to three destinations
□ route long-retention logs to a dedicated bucket rather than raising _Default
□ Data Access audit logs are OFF by default — enabling them everywhere is expensive

# Where the volume is coming from
gcloud logging read 'timestamp>="2026-08-18T00:00:00Z"' --limit=1 >/dev/null
# then use the Logs Dashboard / metric:
#   logging.googleapis.com/billing/bytes_ingested  grouped by log_source

Enable Data Access audit logs deliberately, not globally. They are the highest-volume audit category by a wide margin, and turning them on for every service in every project is one of the reliable ways to multiply a logging bill overnight. Turn them on for the services holding data that matters.

Try it yourself: run the ingestion metric grouped by log source, then add one exclusion for health checks and look again the next day. The size of the drop is usually larger than anyone on the team expects, and it makes the case for the rest of the checklist without an argument.

Common mistake: raising _Default retention to a year to satisfy a compliance requirement that applies to audit logs only. Every debug line, every health check and every application log is now kept — and billed — for a year, to satisfy a rule about a small subset. Route the logs that need long retention into their own bucket and leave _Default short.