Big Data Engineer

We are looking for a skilled Big Data Engineer with 5+ years of experience in building scalable data solutions. The ideal candidate will have strong coding skills (preferably in Python or Java) and experience in managing the end-to-end development lifecycle—from development to deployment in production environments.

Key Responsibilities:

• Design, develop, and maintain robust data pipelines and platforms.
• Build and deploy solutions in cloud environments, either as Cloud Native (Serverless) using S3, Lambda, AWS Batch, ECS, or as Cloud Agnostic using Kubernetes services.
• Implement CI/CD pipelines with GitHub Actions or Jenkins.
• Manage Infrastructure as Code with tools like Terraform.
• Optimize data processing workflows to handle large-scale datasets efficiently.

Preferred Expertise in One or More Areas:

1. Big Data: Experience processing petabytes of data using Spark, Hudi/DeltaLake/Iceberg, Data Catalog, Data Lineage, Glue, EMR.
2. Graph Databases: Experience in ingesting and consuming data in GraphDBs such as Neo4J, AWS Neptune, or JanusGraph.

What We’re Looking For:

• 5+ years of industry experience in data engineering (excluding internships, prototypes, and personal projects).
• Strong coding ability in Python or Java.
• Hands-on experience with cloud-based data solutions and container orchestration.
• Proficiency in CI/CD automation and Infrastructure as Code best practices.
• A passion for building scalable, high-performance data platforms.