Create a business dashboard from Cloud Logging logs using Looker Studio

Search for a command to run...

No comments yet. Be the first to comment.
Background In today's data-driven world, the ability to efficiently query and manage large datasets is crucial for businesses. DuckDB, an in-process SQL OLAP database management system, offers a powerful solution for handling complex analytical queri...
I recently assisted a customer in troubleshooting an issue with their Google Cloud Composer environment and found myself delving deep to uncover the problem. They had 22 DAGs, but only 11 were visible in the UI. It turns out this is a known bug that ...
Background Recently, I assisted a customer who required protection for sensitive data in BigQuery, ensuring access was restricted to a small group of individuals. They wanted to prevent any accidental access by employees. To address this, I recommend...
Background IAM Authentication in Cloud SQL for PostgreSQL lets you manage database user access using Google Cloud Identity and Access Management (IAM) identities instead of traditional PostgreSQL usernames and passwords. This integrates database acce...
Background Directed Acyclic Graphs (DAGs) in Amazon Managed Workflows for Apache Airflow (MWAA) can be triggered in several ways, depending on how much automation and integration you need. While you can manually trigger DAGs using the Airflow UI, we ...
A few months ago, I created a POC for a startup client to visualise their new user registration data. This use case can be interesting for small companies who need a quick way to create a business dashboard. I would like to share it with you.
Assuming you run your business on Google Cloud. Your API is powered by Cloud Function. When there is a new user registration, the Cloud Function instance writes a log to Cloud Logging.

Sinks control how Cloud Logging routes logs. Using sinks, you can route some or all of your logs to supported destinations.
From the Log Sink page, click CREATE SINK. When creating the sink, route the logs to BigQuery:

After creating the sink, it looks like:

A new BigQuery table was created:

Now I put a few logs to Cloud Logging via gcloud command:


On Looker Studio(formerly Data Studio), I built a simple dashboard:

Note that, for testing purposes, I had to switch to minutes so we can see a few bars, but in most cases, the time interval should be hourly or daily.
Using Cloud Logging, BigQuery and Looker Studio, you can quickly build a dashboard to visualise business data, such as user registrations.