Join our dynamic team as an Intermediate Data Engineer and make a lasting impact on the insurance industry! Are you passionate about leveraging data to drive innovation and improve customer experiences? This is your chance to showcase your expertise and be part of an organization that values creativity, collaboration, and cutting-edge technology.
You'll be working with the team towards building and maintaining a world class data platform where numerous data pipelines make data products available for consumption.
Our data pipelines gather data from numerous disperate data sources and bring everything together in a central place - the data platform - where it is made available for downstream consumption. Our data platform serves various types of workloads - from raw data streams, to reporting and aggregation, to advanced analytics.
You will work with a variety of technologies on a day-to-day basis, and we expect you to be hands on. The nature of the job requires constant upskilling and personal development, as our data platform relies on state-of-the-art technologies. Our data platform is built natively on Amazon Web Services (AWS) and integrates to a variety of data sources that are both on-prem and in the cloud.
Duties:
- Your primary duty is to build & maintain our data platform in AWS.
- Write data engineering code and tests using Python and SQL, using a variety of libraries and runtime environments.
- Write infrastructure as code using Terraform and build our deployment pipelines in GitLab.
- Perform code reviews, work with teammates on merge requests, participate in design sessions, etc.
- Participate in the team's Scrum processes and ceremonies.
- Engage with stakeholders to elicit requirements, demonstrate functionality, and drive a data-driven mindset in the organization.
- Operate the data platform and monitor its performance.
- Presenting and communicating Information (written and verbal).
- Analysing.
- Planning and organising.
- Ability to priorities and sequence.
- Delivering results and meeting customer expectations.
- Following instructions and procedures.
- Coping with pressures and setbacks.
- Can work in a team.
- Able to multi-task.
- Sharing knowledge.
- Creating and innovating.
- Persuading and influencing.
- Learning & researching.
- Adapting & responding to change.
- Good programming skills in both Python and SQL.
- Good database design skills and an understanding of various data modelling techniques and approaches.
- An ability to engineer cloud-first data solutions in the Amazon Web Services cloud.
- An understanding of the software development process, with proficiency in Git for version control and build & deployment pipelines in GitLab (or similar) using containerization (Docker).
- Ability to engineer metadata-driven approaches for sustainability and scalability.
- Experience with infrastructure as code, ideally Terraform and Terragrunt.
- An understanding of and exposure to different database technologies.
- Bachelor of Science in Informatics or applicable IT qualification.
- 3 years' experience for Intermediate Data Engineer.
