Senior Data Engineer - (Hadoop+AWS)
Mumbai, India
Nielsen
A global leader in audience insights, data and analytics, Nielsen shapes the future of media with accurate measurement of what people listen to and watch.Job Profile:
To design and develop solutions on AWS for data delivery to downstream systems, andensure timely delivery under agreed SLA.
Description of Duties:
- Design,Develop and implement new solutions on AWS to consume and deliver data as per SLA
- Maintain and support existing applications on AWS platform.
- Liaise with users regarding new and upcoming product features.
- Participate and lead daily team meetings and Sprints.
Qualifications and Experience:
- 5-7 years solid hands-on experience with Hadoop, Spark with Scala on AWS.
- Experience with a range of big data architectures like Hadoop,Spark,Kafka,HBase,Hive,Cassandra or other big data technologies.
- Expertise in designing and developing Spark jobs on AWS environment with varioususe cases
- 2-3 yrs of hands-on experience designing workflows using AWS Airflow or AWS Stepfunctions
- Experience on designing and developing code on Serverless services like AWS Lambda or AWS Glue
- Experience in Tuning and Optimizing Spark jobs
- Knowledge on Terraform templates for Infrastructure provisioning on AWS
- Possess minimum 3+ years of in BI/DW development experience with Data Model Architecture/Design.
- Should have a good understanding of functional programming concepts.
- Good knowledge of Scala with experience of production grade Scala projects.
- Continuous Integration, branching and merging, pair programming, code reviews, unit testing, agile methodologies (Scrum), Design Patterns Knowledge on UNIX/ Shell scripting.
- Hands on experience on NoSQL, preferably Cassandra.
- Knowledge on CI/CD implementation like Ansible/Jenkins for CI/CD pipeline with GitHub/GitLabs repository.
- Knowledge on Scheduling Tools and techniques on Hadoop/EMR.
- Hands-on experience on Java/Web frameworks is an added advantage.
- Excellent written and verbal communication skills.
- Strong analytical and project management skills.
Good to have:
- Proven experience in integrating enterprise software.
- Certification: AWS Cloud Practitioner.
- Knowledge of Multi-Cloud Platform.
Personal Characteristics:
- Creative flair
- Superb attention to detail.
- Great interpersonal and communication skills.
- Confident and dynamic personality.
- Works well independently and in a team.
Do you want to move the industry forward with Nielsen? Our people are the driving force. Yourthoughts, ideas and expertise can propel us forward. Whether you have fresh thinking aroundmaximizing a new technology or you see a gap in the market, we are here to listen and takeaction. Our team is made strong by a diversity of thoughts, experiences, skills, andbackgrounds. You’ll enjoy working with smart, fun, curious colleagues, who are passionateabout their work. Come be part of a team that motivates you to do your best work!
#LI-LM1
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow Ansible AWS Big Data Cassandra CI/CD GitHub Hadoop HBase Kafka Lambda NoSQL Scala Scrum Spark Streaming Terraform Testing
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Marketing Data Analyst jobs
- Open MLOps Engineer jobs
- Open Junior Data Scientist jobs
- Open AI Engineer jobs
- Open Data Engineer II jobs
- Open Senior Data Architect jobs
- Open Sr Data Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Analytics Engineer jobs
- Open Power BI Developer jobs
- Open Manager, Data Engineering jobs
- Open Product Data Analyst jobs
- Open Principal Data Engineer jobs
- Open Business Data Analyst jobs
- Open Data Quality Analyst jobs
- Open Data Manager jobs
- Open Sr. Data Scientist jobs
- Open Data Scientist II jobs
- Open Big Data Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Analyst Intern jobs
- Open Principal Data Scientist jobs
- Open ETL Developer jobs
- Open Azure Data Engineer jobs
- Open Data Product Manager jobs
- Open Business Intelligence-related jobs
- Open Data quality-related jobs
- Open Privacy-related jobs
- Open Data management-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open Deep Learning-related jobs
- Open APIs-related jobs
- Open PyTorch-related jobs
- Open PhD-related jobs
- Open Consulting-related jobs
- Open TensorFlow-related jobs
- Open Snowflake-related jobs
- Open NLP-related jobs
- Open Data warehouse-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs
- Open LLMs-related jobs
- Open DevOps-related jobs
- Open Kubernetes-related jobs