	In Browser
	StumbleUpon
	del.icio.us
	Google
	Google Buzz
	reddit
	LinkedIn

	Facebook
	Twitter
	Linkedin
	E-Mail

Cloud Platforms > Google Cloud Platform (GCP) > Cloud Logging

Cloud Logging

Author: Venkata Sudhakar

Google Cloud Logging is a fully managed, real-time log management service that allows you to store, search, analyze, monitor, and alert on log data from GCP services and applications. It ingests hundreds of petabytes of log data per day.

Key Features:

1. Automatic log ingestion - Automatically collects logs from 150+ GCP services with no configuration.

2. Log Explorer - Powerful query interface using Logging Query Language (LQL) to search and filter logs.

3. Log-based metrics - Create custom metrics from log entries for monitoring and alerting.

4. Log sinks - Export logs to BigQuery, Cloud Storage, or Pub/Sub for long-term retention and analysis.

5. Log retention - Configurable retention from 1 day to 10 years. Default is 30 days for most log types.

The below example shows how to write, query, and export logs using the Cloud Logging API.

import google.cloud.logging
from google.cloud.logging import Client
from google.cloud.logging_v2.handlers import CloudLoggingHandler
import logging

# Initialize Cloud Logging client
client = Client(project="my-project")

# Write structured logs using Python logging library
handler = CloudLoggingHandler(client, name="my-app")
cloud_logger = logging.getLogger("cloudLogger")
cloud_logger.setLevel(logging.INFO)
cloud_logger.addHandler(handler)

# Log structured messages
cloud_logger.info("User login", extra={"json_fields": {
    "userId": "user-123",
    "action": "login",
    "ipAddress": "192.168.1.10",
    "success": True
}})

cloud_logger.error("Payment failed", extra={"json_fields": {
    "orderId": "order-456",
    "amount": 99.99,
    "errorCode": "INSUFFICIENT_FUNDS"
}})

# Query recent error logs
from google.cloud.logging_v2 import _helpers
for entry in client.list_entries(
    filter_='logName="projects/my-project/logs/my-app" AND severity>=ERROR',
    order_by=google.cloud.logging.DESCENDING,
    page_size=10
):
    print(f"{entry.timestamp} [{entry.severity}]: {entry.payload}")

It gives the following output,

2024-01-15T10:05:23Z [ERROR]: Payment failed - orderId: order-456, errorCode: INSUFFICIENT_FUNDS
2024-01-15T09:58:12Z [ERROR]: Database connection timeout - retries: 3

Log Sinks for Export:

BigQuery sink - Export logs to BigQuery for long-term analysis, dashboards, and SQL-based queries.

Cloud Storage sink - Archive logs to GCS buckets for compliance and cold storage at low cost.

Pub/Sub sink - Stream logs in real-time to Pub/Sub for downstream processing, SIEM integration, or alerting.

Send your comments, suggestions or queries regarding this site to [email protected].