Data Engineering Senior Specialist (Databricks/AWS)
Bengaluru-Affluence
Nasdaq is continuously revolutionizing markets and undergoing transformations while we adopt new technologies to develop innovative solutions, constantly aiming to rewrite tomorrow.
With this position we offer :
You will be primarily responsible for taking the defined business requirements and provides analytical, modelling, dimensional modeling and testing to provide best outcomes for design against these customer’s requirements.
Also perform analysis, concentrates on understanding the business needs for data and information. The data engineer translates business information needs into data structures that are adaptable, extensible and sustainable.
Your role and responsibilities:
As a Databricks Data Engineer, you will play a crucial role in designing, building, and maintaining data pipelines within the Databricks Lakehouse Platform. Your expertise will contribute to the success of data-driven initiatives, enabling efficient data processing, analysis, and reporting.
Furthermore, you will:
- Utilize the Databricks Lakehouse Platform to perform data engineering tasks.
- Understand the Lakehouse Platform, its workspace, architecture, and capabilities.
- Implement multi-hop architecture ETL tasks using Apache Spark™ SQL and Python in both batch and incremental paradigms.
- Develop and maintain ETL pipelines using the Medallion Architecture.
- Be able to add new Sources to the Lakehouse platform independently.
- Reviews development of technology platform on AWS cloud and where necessary in house.
- Monitors the design and implementation of the methods to extract data’s from different sources.
- Is responsible for identifying, analyzing and resolving technical problems with Nasdaq Data Platforms and related ecosystems.
- Actively works towards and directs others to ensure the project is being delivered within the assigned timeline and budget
- Oversees the administrative tasks and make sure the team performs them in a timely manner and that information given is precise and complete. Ensure that the administration is done according to regulations, in order to keep the administration accurate and precise.
We expect you to have :
- At least 5-6 years of Data Engineering experience specifically working on Databricks or Snowflake could be preferred.
- Proficiency in SQL and Python for Data Manipulation and transformation.
- Knowledge of Modern Data technologies such as Spark, Parquet and Delta Tables.
- Must have worked on cloud computing platforms like AWS.
- Understanding of data modeling, architecture, and best practices.
- Familiarity with AI/ML Ops in Databricks.
- Education Qualification: Bachelor/Master degree in relevant field or equivalent qualification.
It would be great if you :
- Knowledge on Terraform.
- Any certification in relevant field.
Does it sound like you? What happens now?
As the selection process is ongoing, please submit your application in English as soon as possible. We will get back to you in 2-3 weeks.
What will it be like working here?
Nasdaq is a vibrant and entrepreneurial company where everyone is encouraged to take initiative, challenge status quo, and take intelligent risks. We want everyone to feel welcome and bring their authentic self to work. Every day, we are building a culture where we all feel connected, supported and empowered. We are a Hybrid-first environment that embraces work-life balance and fosters well-being.
What do we offer you?
- Annual monetary bonus.
- An opportunity to become a Nasdaq shareholder
- Employee Stock Purchase Program Nasdaq stocks with a discount
- Health Insurance Program
- Flexible working schedule and hybrid way of work
- Flex day program (up to 6 paid days off a year)
- Internal mentorship program – get a mentor or become one
- Wide selection of online learning resources, e.g., Udemy
Come as You Are
Nasdaq is an equal opportunity employer. We positively encourage applications from suitably qualified and eligible candidates regardless of age, color, disability, national origin, ancestry, race, religion, gender, sexual orientation, gender identity and/or expression, veteran status, genetic information, or any other status protected by applicable law.
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request an accommodation.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Architecture AWS Databricks Data pipelines Engineering ETL Machine Learning Parquet Pipelines Python Snowflake Spark SQL Terraform Testing
Perks/benefits: Career development Flex hours Health care Insurance Salary bonus
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Marketing Data Analyst jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Principal Data Engineer jobs
- Open Data Manager jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Sr Data Engineer jobs
- Open Business Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Data Analyst Intern jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Principal Data Scientist jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open Research Scientist jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs