Manager Data Engineering-DE-Big Data AWS
Publicis Sapient is a digital transformation partner helping established organizations get to their future, digitally-enabled state, both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods, fusing strategy, consulting, and customer experience with agile engineering and problem-solving creativity. United by our core values and our purpose of helping people thrive in the brave pursuit of the next, our 20,000+ people in 53 offices around the world combine experience across technology, data sciences, consulting and customer obsession to accelerate our clients’ businesses through designing the products and services their customers truly value.
As Manager, of Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing Cloud & Non-Cloud based big data solutions for clients. Your role will be focused on delivering high-quality solutions by independently driving design discussions related to below aspects:
- Data Ingestion, Transformation & Consumption,
- Data Storage and Computation Frameworks,
- Performance Optimizations,
- Infrastructure, Automation & Cloud Computing,
- Data Governance & Security
The role requires a hands-on technologist with expertise in Big Data solution architecture and with a strong programming background in Java / Scala / Python, should have experience in creating Data Ingestion pipelines for streaming and batch datasets, creating ETL/ELT data pipelines using distributed computing frameworks like Spark, Strom, Flink, etc, orchestrating data pipelines, should have experience in setting up secure big data platform. You are also required to have hands-on knowledge of at least one of the AWS, GCP, and Azure cloud platforms.
Role & Responsibilities:
1. Provide technical leadership and hands-on implementation role in the areas of data engineering including data ingestion, data access, modeling, data processing, visualization, design, and implementation.
2. Lead a team to deliver high-quality big data technologies-based solutions either on-premise or on Cloud. Manage functional & non-functional scope and quality
3. Help establish standard data practices like governance and address other non-functional issues like data security, privacy, and quality
4. Manage and provide technical leadership to a data program implementation based on the requirement using agile technologies
5. Participate in workshops with clients and align client stakeholders to optimal solutions.
6. Consulting, Soft Skills, Thought Leadership, Mentorship, etc.
7. People management, contributing to hiring and capability building
- Overall 8+ years of IT experience with 3+ years in Data related technologies
- 3+ years of experience in Big Data technologies and expertise of 1+years in data-related Cloud services (AWS / Azure / GCP) and delivered at least 1 project as an architect.
- Mandatory to have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big data solutions either on-premise or on the cloud.
- Expert in Hadoop eco-system with one or more distributions like Cloudera and cloud-specific distributions
- Expert in programming languages like Java/ Scala and good to have Python
- Expert in one or more big data ingestion tools (Sqoop, Flume, NiFI etc), distributed messaging and ingestion frameworks (Kafka,Pulsar, Pub/Sub, etc), and good-to-know traditional tools like Informatica, Talend, etc.
- Expert in at least one distributed data processing framework: Spark (Core, Streaming, SQL), Storm or Flink, etc.
- Should have worked on MPP style query engines like Impala , Presto, Athena, etc
- Should have worked on any of NoSQL solutions like Mongo DB, Cassandra, HBase, etc, or any of Cloud-based NoSQL offerings like DynamoDB, Big Table, etc.
- Should have a good understanding of how to set up Big data cluster security – Authorization/ Authentication, Security for data at rest, and data in Transit.
- Should have a basic understanding of how to manage and set up Monitoring and alerting for Big data clusters. Job Title: Manager – Data Engineering
- Should have worked on any of Orchestration tools – Oozie, Airflow, Ctr-M, or similar.
- Worked on Performance Tuning, Optimization, and Data security
1. Excellent understanding of data technologies landscape/ecosystem.
2. Well-versed with the pros and cons of various database technologies like Relational, NoSQL, MPP, and Columnar databases
3. Good Exposure in development with CI / CD pipelines. Knowledge of containerization, orchestration and Kubernetes engine would be an added advantage.
4. Well-versed in in multi-dimensional modeling like start schema, snowflakes, normalized and de-normalized models
5. Exposure to data governance, catalog, lineage, and associated tools would be an added advantage.
6. Well-versed with Software as a service, Platform as a service, and Infrastructure as a service concept and can drive clients to a decisions
7. Thought Leadership – blogs, keynote sessions, POV/POC, hackathon
8. Certification in either one of the cloud platforms or big data technologies
- Strong analytical and problem-solving skills
- Strong communication skills in verbal, written and visual presentations
- Strong coordination and negotiation skills
- Self-starter who requires minimal oversight
- Ability to prioritize and manage multiple tasks
- Multi geo experience and distributed delivery experience in large programs
- Gender Neutral Policy
- 18 paid holidays throughout the year for NCR/BLR (22 For Mumbai)
- Generous parental leave and new parent transition program
- Flexible work arrangements
- Employee Assistance Programs to help you in wellness and well being
* Salary range is an estimate based on our salary survey 💰
Tags: Agile Airflow Athena AWS Azure Big Data Bigtable Cassandra Consulting Data governance Data pipelines DynamoDB ELT Engineering ETL Flink GCP Hadoop HBase Informatica Kafka Kubernetes MPP NoSQL Oozie Pipelines Pulsar Python Scala Security Spark SQL Streaming Talend
More jobs like this
Sydney, New South Wales, … Sydney, New South Wales, Australia Full TimeSenior Senior-levelUSD 131K - 201K * USD 131K+ *
Machine Learning Engineer - Central Platform Team (MLOps / MLInfra) - Open to remote across ANZAPIs Computer Science Engineering Machine Learning ML models MLOps Nonprofit +5
Career development Fitness / gym Flex hours Parental leave Pet friendly +5
Ho Chi Minh City, … Ho Chi Minh City, Ho Chi Minh … Full TimeSenior Senior-levelUSD 55K - 102K * USD 55K+ *
Senior Manager, Data AnalyticsAgile Business Intelligence Data Analytics Data visualization DevOps Engineering KPIs +2
Flex hours Medical leave Salary bonus Startup environment Team events
Explore more AI/ML/Data Science career opportunities
Find open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general, filtered by job title or popular skill, toolset and products used.
- Open BI Developer jobs
- Open Junior Data Analyst jobs
- Open Staff Data Scientist jobs
- Open Data Science Intern jobs
- Open Director, Data Engineering jobs
- Open Junior Data Engineer jobs
- Open Product Data Analyst jobs
- Open Senior Data Analyst (Bangkok Based, relocation provided) jobs
- Open Power BI Developer jobs
- Open Junior Data Scientist jobs
- Open Associate Data Analyst- Customer Experience Group | Bangkok-based jobs
- Open Head of Data Science jobs
- Open Data Analyst (Remote) jobs
- Open Data Analytics Manager jobs
- Open Lead Data Analyst jobs
- Open Marketing Data Analyst jobs
- Open Big Data Engineer jobs
- Open Machine Learning Scientist jobs
- Open Sr. Data Analyst jobs
- Open Data Manager jobs
- Open Data Analyst (Statistics/Python/BI) (Bangkok-based, relocation provided) jobs
- Open Computer Vision Engineer jobs
- Open Data Scientist (Remote) jobs
- Open Autonomous Vehicle System Test Specialist jobs
- Open Data Engineer Intern jobs
- Open Data management-related jobs
- Open APIs-related jobs
- Open ML models-related jobs
- Open Finance-related jobs
- Open Data quality-related jobs
- Open Airflow-related jobs
- Open Consulting-related jobs
- Open Deep Learning-related jobs
- Open Snowflake-related jobs
- Open PhD-related jobs
- Open Scala-related jobs
- Open Business Intelligence-related jobs
- Open Hadoop-related jobs
- Open Data visualization-related jobs
- Open Kafka-related jobs
- Open Data warehouse-related jobs
- Open Git-related jobs
- Open Docker-related jobs
- Open Kubernetes-related jobs
- Open DevOps-related jobs
- Open TensorFlow-related jobs
- Open PyTorch-related jobs
- Open Streaming-related jobs
- Open NLP-related jobs
- Open NoSQL-related jobs