Senior Big Data Engineer
Careem is the leading technology platform of the Greater Middle East. A pioneer of the region’s ride-hailing economy, Careem is expanding its services across its network to include payments, delivery and mass transportation. Established in July 2012, Careem operates in more than 130 cities across 15 countries, has created more than one million job opportunities in the region and hosts over 30 million users. Careem’s engineering team is growing rapidly, and we are looking for talented engineers to help us in our mission to simplify and improve the lives of people and build a lasting institution that inspires.
Main activities and responsibilities:
- Define the architecture, scope and deliver various Big Data solutions.
- Build and maintain a Central Kafka cluster that serves the whole company.
- Ingest new data to a real time OLAP database using Apache Druid
- Design and build a spark based self service tool that helps data analysts and scientists to process heavy load ETLs
- Implement new feature and maintain a Central Presto cluster that enables thousands of people querying data and creating small ETLs
- Work with streaming technologies such as spark streaming that enable a real time data processing
- Keep data secure with Apache Ranger
- 5+ years of experience as a Big Data Engineer or similar role
- 4+ years of experience with spark and hadoop
- Experience working with Hive, Presto or other querying framworks.
- Experience working with real time data processing using kafka and spark streaming or similar
- 5+ years experience programming and/or architecting a back end language with java and scala
- Experience with distributed computing of large clusters
- Proven experience with AWS technologies, specially s3, EMR, lambda, kinesis, cloudformation.
- Familiarity with Docker and Kubernetes
- Creative and innovative approach to problem-solving
What do we offer you?
- Competitive salary
- Unlimited paid annual leave
- Entrepreneurial working environment
- Flexible working arrangements
- Mentorship and career growth