Senior Data Engineer

End Date Sunday 30 March 2025 We Support Flexible Working – Click here for more information on flexible working options Flexible Working Options Hybrid Working Job Description Summary Builds machine learning prototypes, data pipelines and products through extraction, transformation and analysis of data from internal and external sources. Provides insightful, high quality complex information and advice to stakeholders. Leads small teams in the development of machine learning projects Job Description

• Job Description:

Senior Data Engineer

• Our Purpose:
  • For the Customers:

Support the Group’s and Protection growth ambitions through brilliant decisions and delivery of industry-leading Insurance and Protection services

• For our People:

To build the collective capability of our platform community and improve collaboration and cross team learning in order to deliver on our Transformation technology vision by making it a great place to work where our colleagues feel valued, included and empowered to be the best versions of themselves

• About the Role:

You'll play critical role in designing solutions that can provide continuous flow of value to our customers and will be responsible for:

• Collaborating with Product Owners, Engineering Leads, Solution Architects & Engineers to create the Architectural runway for our Platform teams. Once agreed, our Solution Architects continuously invest in maintaining the Architecture runway underpinning our Platform teams
• Partner with Enterprise Architecture & service providers to elaborate on solutions, their interfaces and validating technology assumptions, evaluating implementation alternatives and software lifecycle risks, assumptions, issues & dependencies
• You’ll provide analysis of options and deliver end-to-end solution designs using relevant architectural building blocks as well as producing designs for features that enable frequent incremental customer value delivery
• Promote adaptive design practises to drive collaboration across Platform teams centring on a common vision with continuous feedback
• Technical design risks or issues and driving resolution through architectural or agile practises
• Participate in Agile ceremonies, elicit requirements, shape & validate designs or prototypes that deliver customer outcomes that align to our target architecture
• Leverage industry best-practise, frameworks, standards & policies in creating opportunities & value for our customers
• Supplies critical technical feedback to the financial investment process in order to obtain funding for key solution opportunities
• Sets guiding principles and solution success criteria for platform capability or feature development across our Platforms
• Provide oversight & champion built in quality, decoupled & modernised architectures
• Support Product Owners & Platform Leads in identifying, articulating and impact assessing software & infrastructure currency risks or issues, offering options & recommendations on action planning & continuously ever-greening solutions/systems within the Platform
• Support Product Owners & Platform Leads on the technical aspect of activities linked with Platform financial estimation, spend or risk management processes in line with the platform roadmap
• Help develop our passionate people promote the architecture subject area through mentoring, coaching and participation in Agile ceremonies
• Highly proficient in their engineering domain. Writes code that is regarded by their peers as high quality
• Proactively identifies problems in our product or platform. Improves areas of the code they work on. Leaves things better than they found them
• Effectively escalates problems that have a wider scope than their team or work, helping to bring it to a successful conclusion

Could this be the right fit for me?

• We are looking for true hands-on experts with large-scale production experience with large

We need a leader in data engineering who understands the challenges of modern architecture

• You’ll build a compelling strategic vision and will understand a wide array of technology implications including people and process, bringing this to your comprehensive view of design
• You'll ideally have previous career experience in software or infrastructure engineering and will be well placed to define and sell your vision for Engineers, Architects, and Product Owners alike
• You must be a keen and comfortable communicator who is happy working with a wide range of technical and non-technical colleagues
• You would be able to produce high quality documentation tailored to the audience which would include senior stakeholders. As a role that can require demanding delivery timeframes, you will need to be self-organised and be able to proactively prioritise your time

If this is you, come and join our inclusive, values-led culture focused on making a difference. Apply today, we’d love to hear from you

• Key Responsibilities:

We're in search of someone who,

• Thrives in acquiring & applying findings to create customer & business value
• Has experiences working in multi-disciplinary product, feature or platform teams
• Is excited at the prospect of changing the status quo, inspiring change with new insights, ideas and encouraging experimentations or market research
• Has established systems engineering or solution architecture experience & appreciation of software engineering methods such as Behaviour & Test Driven Development
• Enjoys communicating & influencing collaborators with little or no technical knowledge, influencing their thinking on the art of the possible through technology
• Can identify appropriate project, architecture, service or risk management methodologies that enable customer value realisation
• Is excited to translate varied technology requirements into relevant solutions
• Is well averse in defining solutions using Agile, Emergent & Intentional architecture methods
• Has substantive experiences defining architectural runways or solution architecture roadmaps for inclusive systems which span adaptive infrastructure domains at Digital-scale
• Has established experiences partnering with other collaborators such as Enterprise, Security, Data & Infrastructure architects to create customer value or address business problems
• Holds proficient experiences leading activities associated with Architectural Governance, Risk Management or Compliance processes
• Has appreciation for Architectural Frameworks such as TOGAF, ArchiMate, Software patterns or similar
• Understands how & where third party products could be appropriate, knowledge of technology purchasing processes & best practise
• Be a technical leader both within your team and outside, coaching and/or mentoring other engineers as appropriate
• Work with the product and engineering leads to achieve the vision and customer outcomes for your Labs, through defining a strategy and designing solutions to deliver our target cloud architecture
• Capable technical leader, who takes ownership of design and delivery and is willing to help grow and guide junior engineers towards appropriate target architectures
• Strong architecture skills with a good working knowledge of microservice design patterns, containerised applications, infrastructure as code in containerised environments, embedding DevOps principles to support delivery at scale and designing solutions at scale

Here's where you'll make a difference:

• Your contribution will be instrumental to our strategy of simplifying, modernising, and de-risking the Group technology landscape, ensuring value for customers and shareholders. This is a key role driving the long-term sustainability and integrity of the organisation
• Our continued commitment to helping Britain recover means that as a colleague you can make a difference to customers, businesses, and communities
• Together we have a key role to play in shaping the bank of the future, whilst the scale and reach of our Group means you'll continue to have opportunities to learn, grow and develop
• We're focused on creating a values-led culture, and our approach to inclusion and diversity means that we all have the opportunity to make a real difference, together
• As part of the Group's commitments as a result of ring-fencing legislation, colleagues based in the Islands are required to be exclusively dedicated to the non-ring-fenced bank and its subsidiaries. This means that colleagues who are based in the Islands would not be able to undertake roles for the Ring-Fenced Bank from their existing location and would need to consider relocation when applying for roles

• Required Skills:

We are looking for hands-on expert level candidates

• with experience in working with large scale data platforms, scaling large scale k8s clusters, performance tuning and NFRs like scalability, security, observability etc. are ingrained in your work
• who have a proven track record with large scale data platforms and have developed large scale data pipelines in production

You will have skillsets to cover below as a minimum;

• Experienced in building and deploying ETL data pipelines with strong debugging experience in Python/PySpark, Cloud Databases preferably on GCP
• Expertise in building real-time and batch based ETL Ingestion & transformation pipelines using Data Flow, DBT, Data Fusion and Data Proc using SQL/No SQL Databases preferred
• Experienced in cloud-based scheduling & orchestration tools and cloud services like GCS, Kubernetes, Kafka, PubSub
• Expertise in developing CICD pipelines using Jenkins/Harness, deploying IAC using Terraform and monitoring using Dynatrace
• Critical thinking skills including analysis of specified problems and issues to find the best technical and/or professional solutions
• Experience with Releases and deployments to production, troubleshooting in production
• Qualifications:
  • 14+ years of relevant experience in design, development and architecting complex software solutions
  • 10+ years of experience working as a hands-on software developer end-to-end solution
  • 6+ year of experience in designing and developing micro-front end and microservices
  • 6+ years of experience working in large financial services; preferably banking industry
  • 6+ years of Systems Architecture and Application design experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
  • 4+ years of experience with Google Cloud/ Azure Cloud/ AWS (or comparable CSP)
  • Computer/Data Science educational background and certifications such as GCP Professional Data Engineer, AWS Solutions Architect, Azure Data Engineer a big plus
• Tech Stack:
  • Primary Skills:

Programming - Python/PySpark, SQL ETL Pipelines (preferred)- Data Flow, DBT/Data Fusion, and/or Data Proc (alternative) - AWS Lambda, AWS Glue, AWS EMR, Azure Data Factory Databases - Big Query, Spanner/Big Table, Cloud SQL (preferred) Or RDS Oracle, Cockroach, Snowflake, Dynamo DB Scheduling/Orchestration: Composer Airflow DAG or any cloud-based scheduler Other Cloud services: GCS/S3, Kubernetes, KMS, Kafka/ PubSub, GitHub CICD pipeline using Jenkins/Harness IAC using Terraform/Harness

• Secondary Skills: Dynatrace/ AppDynamics, Splunk/ Google Cloud Logging, Unix/ Linux