Lead Data Engineer
Hyderabad, Telangana, India
JPMorgan Chase & Co.
Embrace this pivotal role as an essential member of a high performing team dedicated to reaching new heights in data engineering. Your contributions will be instrumental in shaping the future of one of the world’s largest and most influential companies.
As a Senior Lead Data Engineer at JPMorgan Chase within the CCB Connected Commerce Technology Team, you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics in a secure, stable, and scalable way. Leverage your deep technical expertise and problem solving capabilities to drive significant business impact and tackle a diverse array of challenges that span multiple data pipelines, data architectures, and other data consumers.
Job responsibilities
- Provides recommendations and insight on data management, governance procedures, and intricacies applicable to the acquisition, maintenance, validation, and utilization of data
- Designs and delivers trusted data collection, storage, access, and analytics data platform solutions in a secure, stable, and scalable way
- Defines database back-up, recovery, and archiving strategy
- Generates advanced data models for one or more teams using firmwide tooling, linear algebra, statistics, and geometrical algorithms
- Approves data analysis tools and processes
- Creates functional and technical documentation supporting best practices
- Proactively identifies hidden problems and patterns in data and uses these insights to drive improvements to coding hygiene and system architecture.
- Design & develop data pipelines end to end using Spark SQL, Java and AWS Services. Utilize programming languages like Java, Python, NoSQL databases, SQL, Container Orchestration services including Kubernetes, and a variety of AWS tools and services.
- Contributes to software engineering communities of practice and events that explore new and emerging technologies
- Evaluates and reports on access control processes to determine effectiveness of data asset security
- Adds to team culture of diversity, equity, inclusion, and respect
Required qualifications, capabilities, and skills
- Formal training or certification on software engineering concepts and 5+ years of applied experience.
- Hands-on practical experience in system design, application development, testing and operational stability.
- Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages and database querying language.
- Hands-on practical experience in developing spark-based Frameworks for end-to-end ETL, ELT & reporting solutions using key components like Spark SQL & Spark Streaming.
- Proficient in coding in one or more Coding languages - Java, Python
- Experience with Relational and No SQL databases,
- Cloud implementation experience with AWS including:
- AWS Data Services: Proficiency in Lake formation, Glue ETL (or) EMR, S3, Glue Catalog, Athena, Kinesis (or) MSK, Airflow (or) Lambda + Step Functions + Event Bridge
- Data De/Serialization: Expertise in at least 2 of the formats: Parquet, Iceberg, AVRO, JSON-LD
- AWS Data Security: Good Understanding of security concepts such as: Lake formation, IAM, Service roles, Encryption, KMS, Secrets Manager
- Proficiency in automation and continuous delivery methods.
- Proficient in all aspects of the Software Development Life Cycle.
- Solid understanding of agile methodologies such as CI/CD, Applicant Resiliency, and Security
Preferred qualifications, capabilities, and skills
- Snowflake knowledge or experience preferred
- In-depth knowledge of the financial services industry and their IT systems
- Worked with building Data lake, built Data platforms, built Data frameworks, Built/Design of Data as a Service AP
We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow Architecture Athena Avro AWS CI/CD Data analysis Data management Data pipelines ELT Engineering ETL Java JSON Kinesis Kubernetes Lake Formation Lambda Linear algebra NoSQL Parquet Pipelines Python SDLC Security Snowflake Spark SQL Statistics Step Functions Streaming Testing
Perks/benefits: Team events
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Intern jobs
- Open Lead Data Analyst jobs
- Open Power BI Developer jobs
- Open Data Engineer II jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Science Manager jobs
- Open Marketing Data Analyst jobs
- Open Junior Data Scientist jobs
- Open MLOps Engineer jobs
- Open Data Scientist II jobs
- Open Business Intelligence Developer jobs
- Open Business Data Analyst jobs
- Open Product Data Analyst jobs
- Open Data Analytics Engineer jobs
- Open Data Analyst Intern jobs
- Open Sr Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Sr. Data Scientist jobs
- Open Data Engineering Manager jobs
- Open Senior Data Architect jobs
- Open Junior Data Engineer jobs
- Open Big Data Engineer jobs
- Open Research Scientist jobs
- Open Data Quality Analyst jobs
- Open Azure Data Engineer jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Data quality-related jobs
- Open ML models-related jobs
- Open Business Intelligence-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Data visualization-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open NLP-related jobs
- Open PyTorch-related jobs
- Open TensorFlow-related jobs
- Open LLMs-related jobs
- Open APIs-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open Snowflake-related jobs
- Open Consulting-related jobs
- Open Hadoop-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Databricks-related jobs
- Open Airflow-related jobs