Data Engineer

The Company

Metropolis develops advanced computer vision and machine learning technology that make mobile commerce remarkable. Our platform is already deployed in hundreds of mobility facilities and industries with billions in opportunity. We’re building the digital pipes through which the future of mobile commerce will move.

Role

The company is seeking a Data Engineer to help scale data ingestion and processing for our suite of mobility products. As a member of our rapidly growing Data team, you will have the opportunity to work closely with our engineering, analytics, and computer vision leadership to define, build, and maintain data pipelines and data lakes.  You will be expected to design and build libraries and frameworks to accelerate new data pipeline implementation. You will also actively participate in the technical implementation of our data systems, as well as designing the methodology and infrastructure to scale and monitor data for both internal and external customers in a secure and performant fashion.  We are on a mission to empower internal users to make data-backed business decisions quickly and intuitively and you will be essential to us achieving that vision.

Responsibilities 

• Develop and maintain libraries and frameworks to support scalable data pipelines and builds out new pipelines to support the ever-increasing data volume and complexity
• Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
• Collaborate with Analytics, BI, ML, and business teams to improve data models that feed business intelligence & computer vision tools, increasing data accessibility and fostering data-driven decision making across the organization
• Write unit/integration tests, contribute to engineering wiki, and document work
• Keep up with the latest technology trends and strategically evaluates open source and vendor tools

Requirements and Qualifications

• 5+ years job experience writing code
• 3+ years job experience building applications with Scala
• 2+ years job experience with Apache Spark
• 2+ years job experience with any data lake table formats (Apache Hudi, Apache Iceberg, Delta Lake)
• 2+ years job experience using orchestration tools (Airflow, Luigi, etc..)
• Advanced SQL skills
• Strong AWS experience (specifically EMR; generally RDS, S3, ECS, etc..)
• Excellent oral and written communication skills 
• Willingness to learn and teach

Nice to Have

• Building data systems from scratch a huge plus
• Experience with any of the following: 
  • PrestoDB or Trino
  • Apache Ranger or AWS Lake Formation
  • Apache Atlas or Amundsen
  • Apache Kafka or Apache Flink 
  • Containerization (Docker, Kubernetes, ECS, EKS)
  • Data modeling

Our Benefits

• Attractive compensation and generous benefits package (100% Medical, 50% Vision and 50% Dental)
• Company 401(k) match up to 6% of salary
• Paid company snacks and lunches
• Unlimited Paid Time Off (PTO)
• Gym reimbursement program
• 529 savings plan

When you join Metropolis, you’ll join a team of world-class product leaders and engineers, building an ecosystem of technologies at the intersection of parking, mobility, and real estate. Our goal is to build an inclusive culture where everyone has a voice and the best idea wins. You will play a key role in building and maintaining this culture as our organization grows.