TTT-Astra Data Engineer-Senior
Bengaluru, KA, IN, 560016
EY
EY provides consulting, assurance, tax and transaction services that help solve our client’s toughest challenges and build a better working world for all.At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.
Astra Data Engineer (Senior) – Job Description
Skill set
- 3 to 4 years of hands-on experience in data engineering using cloud services like Astra and Databricks.
- Experience in data management and maintenance of NoSQL databases like Astra and Cassandra.
- Thorough knowledge and working experience in developing No SQL data models, scripting and querying in CQL, data mining, clustering, partitioning and segmentation techniques.
- Exposure and working experience in running spark jobs (written in Scala) in Databricks including configuring jobs, monitoring clusters, and troubleshooting job failures.
- Basic understanding on the architecture of applications written in Java or .Net and their interaction with databricks jobs through API.
- Understand and have practical experience with infrastructure technologies and components like cloud services, servers, and networking concepts.
- Proven experience as a data engineer in a similar role involving large data volume applications.
- Understanding of CI/CD release pipelines and code maintenance architectures.
- Overall understanding of SDLC Methodologies and practices, agile working model and processes.
- Strong analytical skills with detailed problem-solving abilities and good communication skills (verbal and written).
Responsibilities
- Work with business and application development teams to analyse and understand business data requirements and translate them into data architectures.
- Partner with Sr. Data architects and infrastructure teams to create platform level database designs and arrive at database platform level decisions.
- Build, test, and maintain the database architecture for a large-scale data processing requirements using technologies such as Astra, Databricks, Apache Cassandra. Configure and administer replication, high availability, and encryption of data.
- Perform daily monitoring activities on the database clusters and spark jobs across all environments in the infrastructure, proactively identify problem areas or red flags and communicate to stakeholders.
- Work with infrastructure teams on maintenance activities, tuning databases, and DBMS to achieve high performance and availability, use data monitoring and analysis tools, tune configuration parameters, alter physical designs, benchmark etc.
- Perform development changes such as data evolutions, data level changes, schema changes and database migrations based on the requests received from the business team and development teams.
- Work with DataStax Support engineers on POCs for new requirements, support questions, investigation and troubleshooting of issues coming up across all environments.
- Participate in design and architecture discussions and recommend efficient data models in terms of performance and management of data.
- Own the release and deployment activities on the database side, track changes made per environment, automate deployments (scripts execution), perform testing on database releases.
- Collaborate with data scientists and architects on the team for debugging issues and providing on-going system maintenance. Carry out systematic and rational analysis to identify the root causes of issues.
- Maintain data security, privacy, ensure systems follow the industry’s best practices, processes, and standards.
- Document database requirements inclusive of data and transaction volumes, scalability, security, and performance requirements.
EY | Building a better working world
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.
Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.
Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile APIs Architecture Cassandra CI/CD Clustering Consulting Databricks Data management Data Mining Engineering Java NoSQL Pipelines Privacy Scala SDLC Security Spark SQL Testing
Perks/benefits: Career development Team events
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Engineer II jobs
- Open Data Science Manager jobs
- Open Marketing Data Analyst jobs
- Open Lead Data Analyst jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Principal Data Engineer jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Business Data Analyst jobs
- Open Data Analyst Intern jobs
- Open Sr Data Engineer jobs
- Open Product Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Research Scientist jobs
- Open Data Quality Analyst jobs
- Open Azure Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Manager, Data Engineering jobs
- Open Junior Data Engineer jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Business Intelligence-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Deep Learning-related jobs
- Open PhD-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open PyTorch-related jobs
- Open NLP-related jobs
- Open TensorFlow-related jobs
- Open APIs-related jobs
- Open Consulting-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Generative AI-related jobs
- Open Snowflake-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs