Senior Data Engineer-Digital Banking Kotak 811-Regional Sales
Bengaluru, Karnataka, India
Kotak Mahindra Bank
Kotak Mahindra Bank offers high interest rate savings account, low interest rate personal loan and credit cards with attractive offers. Experience the new age Personal Banking and Net Banking with Kotak Bank.Job Title: Senior Data Engineer
Job Description
As a Senior Data Engineer, you will play a key role in designing and implementing data solutions @Kotak811.
- You will be responsible for leading data engineering projects, mentoring junior team members, and collaborating with cross-functional teams to deliver high-quality and scalable data infrastructure.
- Your expertise in data architecture, performance optimization, and data integration will be instrumental in driving the success of our data initiatives.
Responsibilities
- Data Architecture and Design:
- Design and develop scalable, high-performance data architecture and data models.
- Collaborate with data scientists, architects, and business stakeholders to understand data requirements and design optimal data solutions.
- Evaluate and select appropriate technologies, tools, and frameworks for data engineering projects.
- Define and enforce data engineering best practices, standards, and guidelines.
- Data Pipeline Development & Maintenance:
- Develop and maintain robust and scalable data pipelines for data ingestion, transformation, and loading for real-time and batch-use-cases
- Implement ETL processes to integrate data from various sources into data storage systems.
- Optimise data pipelines for performance, scalability, and reliability.
- Identify and resolve performance bottlenecks in data pipelines and analytical systems.
- Monitor and analyse system performance metrics, identifying areas for improvement and implementing solutions.
- Optimise database performance, including query tuning, indexing, and partitioning strategies.
- Implement real-time and batch data processing solutions.
- Data Quality and Governance:
- Implement data quality frameworks and processes to ensure high data integrity and consistency.
- Design and enforce data management policies and standards.
- Develop and maintain documentation, data dictionaries, and metadata repositories.
- Conduct data profiling and analysis to identify data quality issues and implement remediation strategies.
- ML Models Deployment[1] & Management (is a plus)
- Responsible for designing, developing, and maintaining the infrastructure and processes necessary for deploying and managing machine learning models in production environments
- Implement model deployment strategies, including containerization and orchestration using tools like Docker and Kubernetes.
- Optimise model performance and latency for real-time inference in consumer applications.
- Collaborate with DevOps teams to implement continuous integration and continuous deployment (CI/CD) processes for model deployment.
- Monitor and troubleshoot deployed models, proactively identifying and resolving performance or data-related issues.
- Implement monitoring and logging solutions to track model performance, data drift, and system health.
- Team Leadership and Mentorship:
- Lead data engineering projects, providing technical guidance and expertise to team members.
- Conduct code reviews and ensure adherence to coding standards and best practices.
- Mentor and coach junior data engineers, fostering their professional growth and development.
- Collaborate with cross-functional teams, including data scientists, software engineers, and business analysts, to drive successful project outcomes.
- Stay abreast of emerging technologies, trends, and best practices in data engineering and share knowledge within the team.
- Participate in the evaluation and selection of data engineering tools and technologies.
- Lead data engineering projects, providing technical guidance and expertise to team members.
Qualifications:
- 3-5 years’ experience with Bachelor's Degree in Computer Science, Engineering, Technology or related field required
- Good understanding of streaming technologies like Kafka, Spark Streaming.
- Experience with Enterprise Business Intelligence Platform/Data platform sizing, tuning, optimization and system landscape integration in large-scale, enterprise deployments.
- Proficiency in one of the programming language preferably Java, Scala or Python
- Good knowledge of Agile, SDLC/CICD practices and tools
- Must have proven experience with Hadoop, Mapreduce, Hive, Spark, Scala programming. Must have in-depth knowledge of performance tuning/optimizing data processing jobs, debugging time consuming jobs.
- Proven experience in development of conceptual, logical, and physical data models for Hadoop, relational, EDW (enterprise data warehouse) and OLAP database solutions.
- Good understanding of distributed systems
- Experience working extensively in multi-petabyte DW environment
- Experience in engineering large-scale systems in a product environment
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Architecture Banking Business Intelligence CI/CD Computer Science Data management Data pipelines Data quality Data warehouse DevOps Distributed Systems Docker Engineering ETL Hadoop Java Kafka Kubernetes Machine Learning ML models Model deployment OLAP Pipelines Python Scala SDLC Spark Streaming
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open MLOps Engineer jobs
- Open Data Science Manager jobs
- Open Data Manager jobs
- Open Lead Data Analyst jobs
- Open Senior Business Intelligence Analyst jobs
- Open Principal Data Engineer jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Data Scientist II jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Product Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Data Analyst Intern jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Junior Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Azure Data Engineer jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Business Intelligence-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open Deep Learning-related jobs
- Open PhD-related jobs
- Open PyTorch-related jobs
- Open TensorFlow-related jobs
- Open APIs-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Generative AI-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open DevOps-related jobs