(Additional locations: San Francisco CA, Sunnyvale CA)
As an engineer on this team, you will be responsible for building and supporting a petabyte-scale data platform in the cloud and providing powerful foundations for Cruise’s ML Data Platform tools, frameworks, and services. Responsibilities include ensuring scalable, transparent, and reliable data ingestion and management; development of fast, robust, and spike-resistant data consumption, data mining, and processing tools for the entire company; and development of orchestration for large-scale post-processing, and computational pipelines.
What you’ll be doing:
-
Lead us in the development, optimization and productionization of the next generation data processing platform using Beam and Spark in the cloud.
-
Build self-serve capabilities to help customers to adopt the next generation data processing platform
-
Use the latest cloud technologies to own, design, implement, and test scalable distributed data systems in the cloud. Champion engineering excellence by continuously improving systems and processes
-
Own technical projects from start to finish, contribute to the team’s product roadmap, and be responsible for major technical decisions and tradeoffs. Effectively participate in team’s planning, code reviews and design discussions
-
Consider the effects of projects across multiple teams and proactively manage conflicts. Work together with partner teams and orgs to achieve cross-organizational goals and satisfy broad requirements
-
Conduct technical interviews with well-calibrated standards and play an essential role in recruiting activities. Effectively onboard and mentor junior engineers and/or interns
What you must have:
-
Experience building a data processing system using Beam / Spark and its ecosystems from the ground up.
-
Experience optimizing those data processing clusters for cost efficiency and performance
-
Experience building serving systems capable of delivering data at high-throughput, low-latency and high QPS in a cost-efficient and spike-resilient manner.
-
Experience building scalable infrastructure on the cloud with Python or Java/Scala (or similar)
-
10+ years working with big data
-
BS, MS or Ph.D. in Computer Science, Electrical Engineering, Mathematics, Physics, or another relevant field; or equivalent real-world experience
-
Passionate about self-driving technology and its potential impact on the world
-
Attention to detail and a passion for seeking truth
-
A track record of efficiently solving complex problems
-
Startup mentality - openness to dealing with unknown unknowns and wearing many hats
Bonus points!
-
Demonstrable expertise in a building end-to-end data ingestion, processing and serving systems at petabyte scale from the ground up
-
Proficiency in writing SQL queries for analytic purposes
-
Relevant publications
The salary range for this position is $183,600 - $270,000. Compensation will vary depending on location, job-related knowledge, skills, and experience. You may also be offered a bonus, and benefits. These ranges are subject to change.
