Data Architect Google Cloud Platform

Data Architect Google Cloud Platform

Posted Today by RKube Inc

Apply
Negotiable
Undetermined
Remote
Remote
Apply
Airflow Apache Airflow AWS Identity And Access Management (IAM) BigQuery Cloud Computing Cloud Storage Cloud Technology Continuous Integration and Continuous Delivery Data Architecture Data Engineering Dataflow Data Governance Data Processing Data Quality Data Security Data Store Generic Programming Git (Version Control System) Google Cloud Google Cloud Composer Google Cloud Platform (GCP) Performance Tuning Pyspark Python (Programming Language) Query Optimisation Scalability Software Development SQL (Programming Language) Version Control Window Function Workflows

Summary: The role of Data Engineer focuses on leveraging Google Cloud Platform to design and implement scalable data pipelines and enterprise-grade data solutions. Candidates should possess hands-on experience with Medallion Architecture and a strong technical background in tools such as BigQuery, PySpark, and Dataflow. The position emphasizes the importance of data governance, performance optimization, and CI/CD practices. This is a remote position aimed at skilled professionals in the data engineering field.

Key Responsibilities:

  • Design, develop, and maintain scalable batch and real-time data pipelines on Google Cloud Platform
  • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing
  • Build high-performance data transformations using Python and PySpark
  • Develop and optimize complex SQL queries for analytical workloads
  • Work extensively with BigQuery for large-scale data processing and performance tuning
  • Develop and deploy pipelines using Cloud Dataflow
  • Orchestrate workflows using Cloud Composer (Apache Airflow)
  • Manage data storage and lifecycle using Google Cloud Storage (GCS)
  • Implement version control and CI/CD pipelines using Git-based tools
  • Ensure data security, governance, and access control using Google Cloud Platform IAM
  • Optimize data solutions for performance, scalability, reliability, and cost-efficiency

Key Skills:

  • Strong hands-on experience with Google Cloud Platform (Google Cloud Platform)
  • Expertise in BigQuery (partitioning, clustering, query optimization)
  • Proven experience implementing Medallion Data Architecture
  • Strong programming skills in Python and PySpark
  • Advanced proficiency in SQL (complex joins, window functions, performance tuning)
  • Hands-on experience with Cloud Dataflow
  • Experience with Cloud Composer (Airflow) for orchestration
  • Experience working with Google Cloud Storage (GCS)
  • Knowledge of version control systems (Git) and CI/CD practices
  • Strong understanding of Google Cloud Platform IAM and cloud security best practices

Salary (Rate): £37.50 hourly

City: undetermined

Country: undetermined

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

seeking a highly skilled Data Engineer with deep expertise in Google Cloud Platform (Google Cloud Platform) and modern data architecture. The ideal candidate will have hands-on experience designing scalable data pipelines, implementing Medallion Architecture, and building robust enterprise-grade data solutions.

This role requires strong technical proficiency in BigQuery, PySpark, Dataflow, and Airflow, along with a solid understanding of cloud data governance, performance optimization, and CI/CD practices.

Key Responsibilities

  • Design, develop, and maintain scalable batch and real-time data pipelines on Google Cloud Platform
  • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing
  • Build high-performance data transformations using Python and PySpark
  • Develop and optimize complex SQL queries for analytical workloads
  • Work extensively with BigQuery for large-scale data processing and performance tuning
  • Develop and deploy pipelines using Cloud Dataflow
  • Orchestrate workflows using Cloud Composer (Apache Airflow)
  • Manage data storage and lifecycle using Google Cloud Storage (GCS)
  • Implement version control and CI/CD pipelines using Git-based tools
  • Ensure data security, governance, and access control using Google Cloud Platform IAM
  • Optimize data solutions for performance, scalability, reliability, and cost-efficiency

Required Skills & Experience

  • Strong hands-on experience with Google Cloud Platform (Google Cloud Platform)
  • Expertise in BigQuery (partitioning, clustering, query optimization)
  • Proven experience implementing Medallion Data Architecture
  • Strong programming skills in Python and PySpark
  • Advanced proficiency in SQL (complex joins, window functions, performance tuning)
  • Hands-on experience with Cloud Dataflow
  • Experience with Cloud Composer (Airflow) for orchestration
  • Experience working with Google Cloud Storage (GCS)
  • Knowledge of version control systems (Git) and CI/CD practices
  • Strong understanding of Google Cloud Platform IAM and cloud security best practices

Preferred Qualifications

  • Experience working with large-scale enterprise data platforms
  • Knowledge of data warehousing and data lake concepts
  • Familiarity with real-time streaming frameworks
  • Experience in data governance and data quality frameworks
  • Exposure to Agile/Scrum methodologies

Apply
Similar Jobs
Loading data...