Data Engineer - Recommendation

About the Team:

Our team supports the Shopee recommendation platform, and strives to provide the best online shopping experience for our users. We focus on big data calculation, real-time distributed data calculation; We collect, manage and analyse petabytes of data daily. Every byte we process generates tangible business value. We are seeking data engineers who are passionate in leveraging cutting-edge data technologies such as Flink Streaming, Spark, Kafka, OLAP Druid to build a solid foundation for our fast-growing business.

Job Description:

• Design, build and maintain real-time streaming system to support timely and accurate ingestion of various types of data (e.g. User behavior, RDS, NoSQL DB, and others) into storage 
• Translate data requirements into scalable technical data service with low latency and high concurrency
• Design, build and maintain batch or real-time data pipeline in production using Hadoop big data technology
• Analyse and improve efficiency, scalability, and stability of the system
• Define and manage service level agreement and data quality for all data sets in allocated areas of ownership

Requirements:

• Minimum Bachelor’s degree in Computer Science or a related technical field
• More than 2 years of experience working  with programming languages, such as Java and Scala
• Proficient in real-time data processing, Flink and Kafka
• Familiar with Hadoop, Spark, and MapReduce; experience with TB batch data processing is a plus
• Familiar with SQL
• Experience in designing and operating of a robust distributed system is a plus
• Understanding data mining or machine learning is a plus
• Passionate to work intimately with data
• Self-motivated and takes ownership