Senior Data Engineer

4 dagar sedan

Göteborgs kommun, Västra Götalands län, Sverige Aurora Engineering AB Heltid

We are looking for an experienced Senior Data Engineer to join our Cloud Platform & Data team. In this role, you will design, develop, and maintain scalable data pipelines and cloud-based data platforms that support secure data sharing and analytics. You will work with modern big data technologies to build reliable data solutions, optimize platform performance, and enable data-driven decision-making across the organization.

Key Responsibilities

  • Design, develop, and maintain scalable batch and streaming data pipelines.

  • Build and optimize ETL/ELT processes for ingesting, transforming, and preparing data.

  • Develop data processing workflows, including cleansing, deduplication, and transformation.

  • Operate, monitor, and maintain cloud-based data platforms in collaboration with infrastructure teams.

  • Perform regression, functional, and security testing following platform upgrades.

  • Ensure data quality, integrity, consistency, and security through validation and monitoring.

  • Troubleshoot and resolve issues related to data pipelines, storage, and integrations.

  • Collaborate with business stakeholders to define data models and support reporting requirements.

  • Provide technical guidance and secure data access to internal data consumers.

  • Design and maintain data models for analytics and reporting.

  • Develop dashboards and reports using Power BI.

  • Document data architectures, pipelines, and technical workflows.

  • Support DevOps, CI/CD, and Infrastructure as Code (IaC) practices for data platform management.

Required Qualifications

  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

  • Minimum 5 years of experience in Data Engineering.

  • Strong experience with Big Data technologies such as Apache Spark, Kafka, Flink, and Data Lakes.

  • Strong understanding of Data Warehousing, ETL/ELT processes, and data modeling.

  • Experience with SQL and Python programming.

  • Hands-on experience with AWS services such as S3, EMR, and Redshift.

  • Experience with Azure Data Factory and Power BI.

  • Experience with Databricks and/or Snowflake.

  • Good knowledge of DevOps, CI/CD pipelines, and software testing.

  • Experience building scalable cloud-based data platforms.

  • Excellent analytical and problem-solving skills.

  • Strong communication skills in English.

Preferred Qualifications

  • Experience with Infrastructure as Code (IaC), such as Terraform or CloudFormation.

  • Experience with automotive or connected vehicle data platforms.

  • Experience with streaming data architectures.

  • Chinese language skills are an advantage.

Technical Skills

  • Data Engineering

  • Apache Spark

  • Apache Kafka

  • Apache Flink

  • Data Lakes

  • ETL / ELT

  • SQL

  • Python

  • AWS (S3, EMR, Redshift)

  • Azure Data Factory

  • Databricks

  • Snowflake

  • Power BI

  • Data Warehousing

  • Data Modeling

  • DevOps

  • CI/CD

  • Infrastructure as Code (IaC)

Key Competencies

  • Data Pipeline Development

  • Cloud Data Platforms

  • Data Quality Management

  • Performance Optimization

  • Problem Solving

  • Analytical Thinking

  • Stakeholder Collaboration

  • Communication

  • Documentation

  • Continuous Improvement