Your First End-to-End Zero Cost Data Pipeline on Google Cloud Platform

Πανεπιστήμιο Μακεδονίας, 156 Εγνατία, Θεσσαλονίκη, 546 36

University of Macedonia - Thessaloniki, Greece

🚀 Hot to Build Your First Data Pipeline with Google Cloud Platform at Zero Cost! 💲

Jun 13, 3:00 – 6:00 PM

0 RSVP'd


Key Themes

Google Cloud

About this event

Join us for a hands-on session where you will learn how to create a simple yet powerful data pipeline using Google Cloud Platform's cost-effective services. This step-by-step guide will walk you through the process of setting up and automating a data pipeline that extracts data from an API, stores it in BigQuery, and visualises it with Looker Studio.

Detailed Agenda

1. Introduction and Setup (10 minutes)

  • Brief overview of Google Cloud Platform services used in the pipeline.
  • Setting up a Google Cloud Project.

2. Creating a Service Account for BigQuery (10 minutes)

  • Step-by-step guide to creating and configuring a service account.
  • Best practices for managing permissions and security.

3. Writing a Python Script for Data Extraction (20 minutes)

  • Introduction to the OpenWeatherMap API for weather data.
  • Live coding session to write a Python script that fetches data from the API and formats it for BigQuery.

4. Generating the requirements.txt File (5 minutes)

  • Explanation of the importance of requirements.txt in managing dependencies.
  • Demonstrate how to generate this file for your project.

5. Deploying the Python Script on Cloud Functions (25 minutes)

  • Detailed instructions on creating and deploying Cloud Functions.
  • Testing and verifying the deployment to ensure it works as expected.

6. Automating the Pipeline with Cloud Scheduler (15 minutes)

  • Setting up Cloud Scheduler to trigger the Cloud Function on a daily basis.
  • Tips on managing schedules and monitoring job executions.

7. Creating a Simple Looker Studio Report (25 minutes)

  • Guide to connecting BigQuery to Looker Studio.
  • Building a basic report to visualise the data collected by your pipeline.

8. Q&A and Troubleshooting (10 minutes)

  • Open the floor for questions from attendees.
  • Common issues and how to resolve them.

**What You'll Learn:**

**Create Google Cloud Projects**: Learn how to set up and manage your projects efficiently.

**Authenticate with Service Account**: Understand how to create and use service accounts for secure access.

**Data Extraction with Python**: Write a Python script to fetch data from a public API.

**Deploy on Cloud Functions**: Automate your Python script using Cloud Functions.

**Automate with Cloud Scheduler**: Schedule your data pipeline to run automatically every day.

**Visualize with Looker Studio**: Create simple yet effective reports to display your data.


To make the most of this session, attendees should:

Have a basic understanding of Python programming.

Set up a Google Cloud account.

**This session is ideal for:**

👨‍💻 Developers and data enthusiasts looking to get hands-on experience with Google Cloud Platform.

👩‍💻 Anyone interested in building and automating data pipelines without incurring costs.

🤖 Professionals wanting to learn how to build time and cost-effective solutions on the cloud.


  • Michalis Chionidis


  • Ilias Papachristos

    Ahead of Tech

    GDG Organizer


  • Efstathios Iosifidis

    GDSC Lead

  • Apostolia Maria Sofianopoulou

    Social Media Ninja


    University of Macedonia

    Instagram Influencer


    Artwork Guru


    University of Macedonia

    Event Orchestrator

Contact Us