About the Role:
We are looking for a Senior Data Engineer to help us continue improving the architecture, enhancing, and maintaining the Data Platform.
Day to Day:
- Design, development, and operations of a Big Data platform in AWS environment that is secure, scalable, robust, and cost effective.
- Ensure scalability of pipelines in both data volume and variety of data sources
- Provide technical leadership, establish standards and patterns for re-use
- Drive data quality across the organization; develop best practices for Data Engineering.
- Partner with architecture, Infosec, and development teams across the enterprise to establish a strong relationship and trust for prioritizing and delivering projects on time
What you bring to the table:
- 5+ years of experience as a data engineer working with a modern tech stack
- Expertise with Spark, SQL, and Python
- Strong knowledge of various AWS services
- Experience with databases and data modeling
- Track record of designing, and deploying high performance systems with reliable monitoring and logging practices
It would be great if you also had:
- Proficiency in Airflow
- Experience with Scala
- Proficiency with Databricks
- Knowledge of container services (Docker/Kubernetes)
