Senior Data Engineer

We are in search of an experienced Senior Data Engineer with a strong background in data engineering. You hold a degree in Computer Science, Statistics, Informatics, Information Systems, or a closely related quantitative field.

Your key qualifications:

- You are well-versed in SQL with a proven track record of crafting complex queries and managing various relational databases.

- Demonstrated expertise in building and optimizing data pipelines and architectural solutions for large and complex datasets.

- Proficient in conducting root cause analyses to resolve specific business questions and identify opportunities for process enhancement within both internal and external data and processes.

- You have a history of strong analytical skills with a successful track record of manipulating, processing, and extracting value from extensive, diverse, and unstructured datasets.

- Experience in developing processes that facilitate data transformation, manage data structures, metadata, dependencies, and workloads efficiently.

- You possess a working knowledge of message queuing, stream processing, and highly scalable data storage solutions.

- You have substantial experience in collaborating with cross-functional teams in a dynamic and fast-paced environment.

We are seeking you, someone who is passionate about data engineering, enthusiastic about data analysis, and adept at creating efficient systems that derive maximum value from data. You are known for your proactive problem-solving skills, innovation, and effective communication, all of which make you a key contributor to any organization's data-driven success.

Key Responsibilities:

- Develop and maintain Data Lake services in AWS.

- Create and manage environments for advanced analytics, machine/deep learning, and AI.

- Implement best practices in continuous integration and delivery.

- Pre-process, clean, and structure data to enable data exploration and advanced analytics/Machine Learning activities.

- Utilize modern data preparation, integration, and AI-enabled metadata management tools and techniques.

- Assemble large, complex data sets that meet functional and non-functional business requirements.

- Perform intelligent sampling and caching.

- Recommend and implement automation in existing and future integration flows.

- Enhance data quality by advancing data acquisition, testing, and tooling.

- Automate system installation and configuration.

- Contribute to the Roadmap and other improvements.

- Provide runtime support for services.

Software/Tools Experience:

- Proficiency in big data tools (Hadoop, Spark, Kafka, etc.).

- Familiarity with relational SQL and NoSQL databases, including Postgres and MongoDB.

- Expertise in data pipeline and workflow management tools (Step Function, Airflow, etc.).

- Knowledge of AWS CI/CD tools.

- Experience with AWS cloud services (EC2, EMR, RDS, Redshift, AWS Lambda, Glue, Athena, Kinesis).

In this dynamic environment, you'll be situated in the core of a bustling hub dedicated to cutting-edge technologies such as autonomous driving, electromobility, digitalization, and IoT. This is a consulting opportunity with one of our clients.