Data Engineer - Recommendation

Singapore

About the Team:

Our team supports the Shopee recommendation platform, and strives to provide the best online shopping experience for our users. We focus on big data calculation, real-time distributed data calculation; We collect, manage and analyse petabytes of data daily. Every byte we process generates tangible business value. We are seeking data engineers who are passionate in leveraging cutting-edge data technologies such as Flink Streaming, Spark, Kafka, OLAP Druid to build a solid foundation for our fast-growing business.

Job Description:

  • Design, build and maintain real-time streaming system to support timely and accurate ingestion of various types of data (e.g. User behavior, RDS, NoSQL DB, and others) into storage 
  • Translate data requirements into scalable technical data service with low latency and high concurrency
  • Design, build and maintain batch or real-time data pipeline in production using Hadoop big data technology
  • Analyse and improve efficiency, scalability, and stability of the system
  • Define and manage service level agreement and data quality for all data sets in allocated areas of ownership

Requirements:

  • Minimum Bachelor’s degree in Computer Science or a related technical field
  • More than 2 years of experience working  with programming languages, such as Java and Scala
  • Proficient in real-time data processing, Flink and Kafka
  • Familiar with Hadoop, Spark, and MapReduce; experience with TB batch data processing is a plus
  • Familiar with SQL
  • Experience in designing and operating of a robust distributed system is a plus
  • Understanding data mining or machine learning is a plus
  • Passionate to work intimately with data
  • Self-motivated and takes ownership
Job region(s): Asia/Pacific
Job stats:  5  1  0
  • Share this job via
  • or

Explore more AI/ML/Data Science career opportunities