Senior Data Engineer
Trickle Up
We are looking for a Senior Data Engineer to design, build, and maintain enterprise-scale data platforms on Azure. You will own end-to-end data pipelines from raw ingestion to curated, analytics-ready gold layers while driving data quality standards, mentoring junior engineers, and collaborating directly with product and AI teams.
Requirements
Pipeline Design & Architecture
▸ Architect and manage scalable ETL/ELT pipelines using Medallion Architecture (Bronze → Silver → Gold)
▸ Design ADF pipelines, data models, and schemas optimised for performance and maintainability
▸ Build near real-time and batch ingestion solutions across structured and semi-structured sources
Data Governance & Quality
▸ Implement data quality checks, validation frameworks, and lineage tracking across pipeline layers
▸ Enforce security controls, access policies, and compliance requirements on ADLS Gen 2 and Azure SQL
▸ Monitor pipeline SLAs, diagnose performance bottlenecks, and own incident resolution
Engineering Excellence
▸ Drive CI/CD adoption for data pipelines using Azure DevOps or equivalent tooling
▸ Define and enforce coding standards, peer-review practices, and documentation norms
▸ Contribute to Al agent development initiatives as a data platform subject-matter expert
Collaboration & Mentorship
▸ Partner with data scientists, analysts, and product managers to translate requirements into reliable data products
▸ Mentor junior engineers through code reviews, design discussions, and structured knowledge sharing
▸ Participate in Agile ceremonies and contribute to sprint planning, backlog refinement, and technical estimation
EXPERIENCE & REQUIREMENTS
Must-Have
▸ 5-6 years of hands-on Data Engineering experience in production environments
▸ Deep expertise in Azure ecosystem — ADLS Gen 2, ADF, Azure SQL Database, and Delta Lake
▸ Strong command of PySpark, Python, and SQL for large-scale data transformation
▸ Proven track record building and operating Medallion Architecture (Bronze/Silver/Gold) on Azure
▸ Experience designing data warehouses and data lake solutions at enterprise scale
▸ Familiarity with CI/CD pipelines, version control (Git), and DevOps practices in a data context
Nice-to-Have
▸ Exposure to AWS or GCP in addition to Azure
▸ Experience supporting or building Al/ML data pipelines and feature stores
▸ Knowledge of streaming frameworks — Kafka, Event Hubs, or Spark Structured Streaming
▸ Azure certifications (DP-203, AZ-900 or equivalent)
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
Junior Software Developer - TJ / 1854396
Design Coordinator ( Architect / Draftsman)
Data Protection Expert