Senior Data Engineer

As a Senior Data Engineer, you will play an important role in the Data Platform team who are responsible for building and maintaining a scalable, robust, self-service analytics platform.

What do we expect from you

• Experience working with orchestration tool (ex: Airflow)
• Experience working with a large data set
• Experience building complex ETL pipelines
• Experience working with cloud providers such as GCP, AWS or Azure
• Strong programming skills in Spark with Scala and Python
• Experience working with CI/CD tools like Jenkins and Git
• Strong understanding of Software Engineering practices and principles
• Excellent problem solving and communication skills.
• Self-motivated and have a proven ability to take initiative to own the problems that come up and solve them.

What will you work on

• Maintaining and improving the ingestion pipeline to reliably deliver billions of events daily in defined SLA
• Provide support for all teams in building and optimizing their complex pipelines
• Work closely with other teams to identify pain points and problems around the platform
• Develop new tools and frameworks to improve the data platform
• Facilitate company wide to be data-driven
• Work in close collaboration with data scientists and data analysts to help support their work go to production
• Work with vast projects such as building an ML platform and streaming use cases
• Setup best practices and processes around software and data development
• 
  

It would be great if you also have

• Working with messaging system like Kafka
• Knowledge in Kubernetes
• Hands on experience with any streaming platform
• Experience managing data warehouse in BigQuery or Redshift