Data Engineer

We are looking for an experienced data engineer with a deep understanding of the requirements needed to support multiple data product teams with competence and process onboarding. The team develops features for the framework that builds an internal platform that enables other product teams to onboard, structure and process data from multiple source systems at xx Group. The data is later consumed by various products with AI and analytical focus. The platform is cloud native and the team is focusing on scaling and robustness of the platform to cater for our consumers’ needs.

The team own the development life cycle, and we aim to continuously bring value to the product teams who will use this new platform. The team is autonomous, and we hope that you appreciate that. We also think it is important to have a hands-on mindset and thinking that no challenge is too big with the right support and knowledge at hand.

We are looking for you who:

• Are passionate about excellence in data platform development; scalability and developing functionality in modules that will be used throughout the platform and by other teams.

• Have significant experience in developing backend solutions within Agile teams using DevOps strategies. Your knowledge in this area provides you with an outstanding view on different technology stacks and architecture paradigms.

• Understand the challenge of design and development large scale distributed system.

• Have an open mindset and used to giving and receiving feedback on suggested improvements as part of your daily work.

• Use your data engineering skills to build our new internal platform for large-scale data processing needs in a reliable and robust way.

• Can identify long-term impacts on quality, security and cost and using common best practices for testing, deployment and run and constantly analyse and benchmark for bottlenecks and plan execution improvements.

• Are thrilled to learn about new technology and pass on this knowledge to your colleagues.

• Have a hands-on approach when building solution prototypes when needed

Competence requirements:

Skills and abilities:

• Expert in Python programming.

• Expert in batch data processing using Apache Airflow.

• Expert skills in working with distributed processing platforms (Apache Spark & Databricks).

• Excellent skills in SQL programming.

• Excellent skills in working with Azure Data Lake Gen 2.

• Proficient experience of building cloud-based solutions in Azure.

• Skilled in software engineering practices like design patterns, unit testing, source control, CI/CD, etc.

• Experience with stream processing & NoSQL Databases (preferably CosmosDB) is a bonus.

• Experience with Infra as a Code tools like Terraform is good to have.

• Experience with Docker & Kubernetes is a bonus.

• Fluent in English in speech and writing is required.