AWS Architect-Data Engineer

We are seeking an experienced AWS Architect - Data Engineer with strong technical expertise and leadership capabilities to design and implement robust, scalable data solutions. The ideal candidate will have deep experience in data engineering technologies and cloud platforms, coupled with a proven ability to lead and mentor a team.

Responsibilities:

• Design and implement scalable data processing pipelines using Apache NiFi, Apache Spark, and Databricks.
• Develop and manage workflows with Apache Airflow to ensure efficient data transformation and loading processes.
• Utilize AWS services including Lambda, EMR, EventBridge, and Redshift for comprehensive cloud-based data solutions.
• Design and maintain robust event buses, integrating Apache Kafka for real-time data streaming.
• Manage and optimize data storage and analytics using AWS Redshift and Delta Lake for improved data reliability and performance.
• Develop complex SQL queries for data extraction, transformation, and loading (ETL) processes.
• Write clean, efficient, and well-documented Python code.
• Build and maintain APIs for data integration and accessibility.
• Collaborate with cross-functional teams to understand data needs and deliver scalable and efficient data solutions.
• Lead and mentor a team of data engineers, promoting best practices in data engineering and handling technical challenges.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Minimum of 5 years of experience in data engineering with a strong track record in a leadership role.
• Extensive experience with NiFi, Spark, Airflow, AWS Lambda, EMR, EventBridge, Kafka, Redshift, Databricks, Snowflake and Delta Lake.
• Strong proficiency in SQL and Python.
• Experience in building and optimizing APIs for data operations.
• Solid understanding of data warehousing concepts, ETL processes, and big data technologies.
• Strong problem-solving and analytical skills.
• Excellent leadership and communication skills.

Preferred Skills:

• Additional experience with other AWS services.
• Knowledge of machine learning and data science methodologies.