Principal Consultant - Data Analytics

We are seeking a highly skilled and motivated Data Engineer to join our team. The ideal candidate will have a strong background in building and optimizing data pipelines, managing distributed data systems, and working with cloud platforms.

Responsibilities:

• Design, develop, and maintain scalable and reliable ETL/ELT pipelines using Apache Spark and Airflow.
• Optimize data workflows for performance, scalability, and fault tolerance.
• Implement and manage streaming data pipelines using Apache Kafka.
• Ensure low-latency data ingestion and processing for real-time analytics.
• Build and manage data solutions on Google Cloud Platform (GCP), leveraging tools such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
• Ensure data security, compliance, and performance in the cloud.

Requirements:

• Proficiency in Apache Spark for distributed data processing.
• Hands-on experience with Apache Kafka for building streaming data pipelines.
• Strong knowledge of Apache Airflow for orchestrating workflows and scheduling tasks.
• Expertise in Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
• Solid understanding of SQL and Python (or other programming languages like Java/Scala).
• Familiarity with data modeling, schema design, and ETL/ELT processes.
• Knowledge of real-time data processing concepts and tools.