You have experience with:
- several programming languages (Python, Scala, Java, etc.)
- AWS services such as EMR, Lambda, S3, Athena, Glue, IAM, RDS, etc.
- orchestration tools such as Airflow, Luiji, Azkaban, Cask, etc.
- streaming data processing frameworks like Kafka, KSQ, and Spark Streaming
- a diverse set of databases like MongoDB, Cassandra, Redshift, Postgres, etc.
- different storage formats such as Parquet, Avro, Arrow, and JSON
- data processing frameworks like Spark
- Git and Github
- CI/CD Pipelines
- Constantly think of ways to squeeze better performance out of a data platform
- Plan effective data storage, security, sharing, and publishing within an organization
- Design boilerplate architecture that can abstract underlying technology from end users
- Design, manage, and test disaster recovery procedures for a variety of data platforms
- Value code simplicity and performance
- Obsess over data: everything needs to be accounted for and be thoroughly tested
- Build great things alone, but the greatest things in collaboration with others
- You are deeply familiar with Spark and/or Hive
- You have expert experience with Airflow
- You understand the intricacies between different storage formats like Parquet, Avro, Arrow, and JSON
- You are familiar with deployment and configuration tools such as Kubernetes, Drone, and Terraform
- You have experience building microservices
- You’ve built an end-to-end production-grade data platform that runs on AWS
- You have experience building a machine learning platform using tools like SparkML, Tensorflow, Scikit-Learn, etc.
As a Data Platform Engineer, you will:
- Build a large-scale batch and real-time platform that will make data pipelines seamless and scalable
- Help drive best practices in continuous integration and delivery
- Help drive optimization, testing, and tooling to improve data platform quality
- Collaborate with other software engineers, machine learning experts, and stakeholders, taking learning and leadership opportunities that will arise every single day
- In three months you will have familiarized yourself with much of our data platform, be making regular contributions to our codebase, will be collaborating regularly with stakeholders to widen your knowledge and helping to resolve incidents and respond to user requests
- In six months you will have successfully investigated, scoped, executed, and documented a small to medium sized project and worked with stakeholders to make sure their data needs are satisfied by implementing improvements to our platform
- In a year you will have become the key person for several projects within the team and will have contributed to the the data platform’s roadmap. You will have made several sizable contributions to the project and are regularly looking to improve the overall stability and scalability of the architecture
*MongoDB is an equal opportunities employer*
To apply for this job please visit www.mongodb.com.
Please mention you found this job on ai-jobs.net to help us get more companies to post here 🙂