Lead Data Engineer

Position: Lead Data EngineerLocation: Gurgaon (Hybrid), Bangalore (Hybrid)Accepted Notice Period: Immediate – 30 daysYears of Experience: 6 – 8 YearsOpportunity Type: Full time
**Note: This position is with one of our hiring partners

Skills Required

• Extensive experience with Hadoop ecosystem technologies such as Hadoop, Hive, and Pyspark/Spark
• Proficiency in programming languages like Python, Java, and Unix scripting for data engineering tasks
• Strong understanding of data engineering principles, best practices, and design patterns
• Hands-on experience in designing, building, and optimizing large-scale data pipelines and ETL processes
• Ability to lead and mentor a team of data engineers, providing technical guidance and expertise
• Knowledge of cloud platforms like AWS/GCP/Azure is highly desirable
• Excellent problem-solving skills and ability to troubleshoot complex data issues
• Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams

Responsibilities

• Lead the design, development, and implementation of data solutions using Hadoop, Hive, and Spark
• Architect and build scalable and reliable data pipelines to support various data processing and analytics requirements
• Provide technical leadership and mentorship to the data engineering team, ensuring best practices and standards are followed
• Collaborate with stakeholders to gather and analyze requirements, and translate them into technical specifications
• Optimize and tune data workflows for performance, scalability, and efficiency
• Ensure data quality, integrity, and security across all data processes and systems
• Stay updated with the latest trends and technologies in the field of data engineering, and drive continuous improvement initiatives
• Drive innovation and contribute to the development of new data products and services

About Pesto Tech:Pesto is where software developers go to build their career path for the next 5 years. We don't just offer jobs; we provide unparalleled opportunities for your growth and success in the dynamic landscape of Tech Jobs.