Data Engineer – Google Cloud Platform

Job Summary

KANINI is seeking a highly skilled Data Engineer with deep expertise in Google Cloud Platform (Google Cloud Platform) and modern data architecture. The ideal candidate will have hands-on experience designing scalable data pipelines, implementing Medallion Architecture, and building robust enterprise-grade data solutions.

This role requires strong technical proficiency in BigQuery, PySpark, Dataflow, and Airflow, along with a solid understanding of cloud data governance, performance optimization, and CI/CD practices.

Key Responsibilities

• Design, develop, and maintain scalable batch and real-time data pipelines on Google Cloud Platform

• Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing

• Build high-performance data transformations using Python and PySpark

• Develop and optimize complex SQL queries for analytical workloads

• Work extensively with BigQuery for large-scale data processing and performance tuning

• Develop and deploy pipelines using Cloud Dataflow

• Orchestrate workflows using Cloud Composer (Apache Airflow)

• Manage data storage and lifecycle using Google Cloud Storage (GCS)

• Implement version control and CI/CD pipelines using Git-based tools

• Ensure data security, governance, and access control using Google Cloud Platform IAM

• Optimize data solutions for performance, scalability, reliability, and cost-efficiency

Required Skills & Experience

• Strong hands-on experience with Google Cloud Platform (Google Cloud Platform)

• Expertise in BigQuery (partitioning, clustering, query optimization)

• Proven experience implementing Medallion Data Architecture

• Strong programming skills in Python and PySpark

• Advanced proficiency in SQL (complex joins, window functions, performance tuning)

• Hands-on experience with Cloud Dataflow

• Experience with Cloud Composer (Airflow) for orchestration

• Experience working with Google Cloud Storage (GCS)

• Knowledge of version control systems (Git) and CI/CD practices

• Strong understanding of Google Cloud Platform IAM and cloud security best practices

Preferred Qualifications

• Experience working with large-scale enterprise data platforms

• Knowledge of data warehousing and data lake concepts

• Familiarity with real-time streaming frameworks

• Experience in data governance and data quality frameworks

• Exposure to Agile/Scrum methodologies