Data Engineer

This is what we'd like you to do: 
  • Manage the Extract, Transform and Load (ETL) processes of the company to ensure they remain efficient and robust
  • Understand the data warehousing and ETL needs of the company to help identify weaknesses in the current architecture and design improvements
  • Upgrade the architecture of our in-house ETL platform as per roadmap
  • Mentor more junior members of the team, moving into a formal Lead role in time
  • Identify and resolve data and reporting issues in a timely manner
  • Help to improve the efficiency and reliability of data systems and processes by identifying areas that would benefit from automation, including testing
  • Work with other platform Analysts and Managers to support the introduction of new technologies and data streams
This is what you'll have done already:
  • Excellent understanding of database and data warehouse technologies, including standard row RDBMS, columnar RDBMS and some form of NoSQL (e.g. key-value, document, graph stores)

  • Strong experience developing ETL scripts in Apache Airflow or similar ETL platform

  • Strong experience of Python 3 including working on enterprise-scale systems

  • Experience using Google BigQuery or similar technology, including schema design

  • Experience using MySQL, including schema design

  • Some experience mentoring or training junior team members

  • Good understanding of Google Cloud Platform or AWS including setting up compute and storage instances from scratch

  • Experience of at least one commercial BI platform

  • Experience writing code which consume 3rd party APIs at scale

  • Ideally some familiarity with machine learning techniques such as logistic regression or random forests

  • Computer Science or Maths degree