Background In today's data-driven world, the ability to efficiently query and manage large datasets is crucial for businesses. DuckDB, an in-process SQL OLAP database management system, offers a powerful solution for handling complex analytical queri...
Background Recently, I assisted a customer who required protection for sensitive data in BigQuery, ensuring access was restricted to a small group of individuals. They wanted to prevent any accidental access by employees. To address this, I recommend...
BigQuery has a useful feature which allows the user to create external table with data on Google Sheets. It is very convenient because BigQuery users can query the data from Google Sheets directly. However, as a data engineer, you may need to build p...
Recently, I worked with a client who wanted to export data from Cloud SQL to GCS daily. They tried Cloud SQL Serverless Export orchestrated by Cloud Composer but found it is pretty expensive for their use case. For the context, with Serverless Export...
"Oops, I accidentally deleted my BigQuery Dataset! What should I do!?" If this is you at the moment, don't panic. You are in good hands now! In this tutorial, I'll work through a lab, in which I deleted a BigQuery Dataset and then recovered it with a...
A few months ago, I created a POC for a startup client to visualise their new user registration data. This use case can be interesting for small companies who need a quick way to create a business dashboard. I would like to share it with you. Logs As...
