Senior Data Engineer

Senior Data Engineer

• Role type: Hybrid 3 days in office, 2 remote
• Location: Pretoria or Cape Town

We are seeking skilled Senior Data Engineers to join our agile team, responsible for designing and implementing data solutions that collect, process, store, and transform raw, unstructured data into formats that can be consumed by downstream teams.

As a Data Engineer, you will simplify access to services and resources, govern and maintain a Data Lake platform, and ensure that data systems meet regulatory commitments while being reliable and efficient.
Key responsibilities:

• Work collaboratively in an agile data engineering team
• Design and implement scalable, end-to-end database solutions
• Ensure compliance with data governance and management frameworks
• Partner with stakeholders to align database structures, programs, and applications with business requirements
• Monitor adherence to data architectural frameworks and required standards and methodologies
• Optimize data pipelines, data modeling, and data warehousing solutions
• Utilize Azure Databricks, AWS, and real-time data streams
• Script data pipelines for scheduled movement of data
• Visualize data with PowerBI/Tableau and perform Exploratory Data Analysis (EDA)

Key skills:

• Experience in Azure Databricks, data governance and management frameworks, BI and warehouse design
• Familiarity with Apache Kafka, Storm, Flume, AWS Cloud Development Kit (CDK), JSON, CSV, and raw data
• Expertise in Python (PySpark), C#, PowerShell, Apache Spark, Kafka, Scala, Splunk, Elk Stack, SQL, and NoSQL
• Knowledge of data modeling, data cleansing, data pipelines, data warehousing solutions, and Terraform

Qualifications:

• 3-year tertiary qualification BSC in Computer Science/Information Systems
• 4+ years senior data engineering experience

If you are a self-motivated Data Engineer who thrives in a fast-paced environment, we encourage you to apply for this exciting opportunity to help shape the future of our data-driven organization.