Monitoring with Cloud Monitoring: Metrics, Dashboards, and Alerting

Cloud Monitoring Overview and Workspaces

Metrics Fundamentals: Types, Labels, and Resources

Exploring Metrics for Compute Engine, GKE, and Cloud Run

Dashboards and Charts: Building a Single Pane of Glass

Thought Exercise: Design a Dashboard for an E-commerce App

Imagine you operate a small e-commerce platform running on Google Cloud:

• Frontend: Cloud Run service `store-frontend` in region `us-central1`.
• Backend: GKE cluster `prod-gke` running a `cart-service` and `orders-service`.
• Database: Cloud SQL for PostgreSQL.

You want a single dashboard for on-call engineers to quickly see if users can browse, add to cart, and place orders.

Your task: On a sheet of paper or in a note, sketch what widgets you would add. Use the prompts below.

1. Cloud Run `store-frontend`

• Which 3 metrics would you chart to detect user-facing issues?
• How would you group or filter them (by service, revision, region)?

1. GKE `cart-service` and `orders-service`

• Which metrics would reveal if these services are overloaded or failing?
• Would you group by `namespace`, `podname`, or `responsecode`?

1. Cloud SQL

• Which basic metrics would tell you if the database is the bottleneck (e.g., CPU, connections, latency)?

1. Summary row

• What 2–3 scorecards would you add at the top as “red/green” indicators for management?

After you sketch, compare against this checklist:

• Do you cover traffic (request count/QPS)?
• Do you cover latency (p95 or p99)?
• Do you cover errors (4xx/5xx rates)?
• Do you cover resource saturation (CPU, memory, DB connections)?

Use this exercise to practice thinking like an on-call engineer. The exam often describes these kinds of scenarios and asks what to monitor or where to look first.

Uptime Checks and SLOs: From Basic Health to Reliability Targets

Alerting Policies and Notification Channels

Quiz: Uptime Checks vs Metric Alerts

Test your understanding of uptime checks and alerting policies.

Custom Metrics and Integration with GKE and Cloud Run

Code Example: Writing a Custom Metric (Python)

This simplified Python example shows how a Cloud Run service or GKE pod might write a custom counter metric to Cloud Monitoring. In practice, you would handle authentication using a service account attached to the workload.

A service account is a special kind of account used by an application or compute workload, not a person, to make authorized API calls and access Google Cloud resources.

```python

from google.cloud import monitoring_v3

from google.api import metricpb2 as gametric

from google.api import monitoredresourcepb2

import time

projectid = "YOURPROJECT_ID"

client = monitoring_v3.MetricServiceClient()

projectname = f"projects/{projectid}"

Define the custom metric type

metrictype = "custom.googleapis.com/orders/createdcount"

series = monitoring_v3.TimeSeries()

series.metric.type = metric_type

series.resource.type = "global" # or "cloudrunrevision", "k8s_container", etc.

Optional labels

series.metric.labels["env"] = "prod"

Set the data point value (e.g., 1 order created)

point = monitoring_v3.Point()

now = time.time()

interval = monitoring_v3.TimeInterval()

interval.end_time.seconds = int(now)

interval.end_time.nanos = int((now - int(now)) 10*9)

point.interval.CopyFrom(interval)

For a counter, use INT64

point.value.int64_value = 1

series.points.append(point)

Write the time series data

client.createtimeseries(name=projectname, timeseries=[series])

print("Wrote 1 data point to", metric_type)

```

On the exam, you will not write code, but you should recognize that:

• Custom metrics are pushed via the Monitoring API.
• Workloads authenticate using a service account.
• Once written, custom metrics appear in Metrics Explorer under the `custom.googleapis.com` namespace.

Quiz: Choosing the Right Monitoring Approach

Check your understanding of metrics, dashboards, and alerts in a realistic scenario.

Troubleshooting Scenario: Slow Cloud Run Service

Work through this mental troubleshooting exercise to practice using Cloud Monitoring data.

Scenario

A Cloud Run service `image-resizer` is used by a mobile app. Users report that image uploads sometimes take 10–15 seconds. Autoscaling seems to work, and there are no obvious errors.

Using only Cloud Monitoring (no code changes), think through these questions:

1. Which metrics do you check first?

• Hint: Consider request latency distributions (p95, p99), request count, and concurrency metrics for `cloudrunrevision`.

1. How do you confirm if the issue is global or regional?

• Hint: Check labels like `location` or use filters by region in Metrics Explorer.

1. How do you distinguish between app slowness and downstream dependency slowness (like Cloud Storage or a database)?

• Hint: Look at metrics for those dependencies (for example, Cloud Storage latency, Cloud SQL CPU/connection usage) on their own dashboards.

1. What alerting policy could you create to catch this faster next time?

• Hint: Consider an alert on p95 latency above a threshold (for example, 2 seconds) for a few minutes, filtered to this service.

Write down your answers, then compare to this reference checklist:

• Checked Cloud Run request latency (p95/p99) and request count.
• Compared latency across regions or revisions using labels.
• Checked metrics for downstream services to see if they spike at the same time.
• Designed an alert on latency, not just CPU or error rate.

Being able to reason through these steps is exactly what the exam tests in scenario questions.

Key Term Review: Cloud Monitoring Essentials

Use these flashcards to reinforce the core concepts before moving on.