Middle Data Engineer
Canada, USA
Exadel
Advance your business through technology and pave the way to becoming a digital leader with Exadel, an enterprise software development and consulting company.We are looking for talented Middle Data Engineer to join our international team of professionals!
Work at Exadel - Who We Are:
Since 1998, Exadel has been engineering its own software products and custom software for clients of all sizes. Headquartered in Walnut Creek, California, Exadel currently has 2700+ employees in development centers across the Americas, Europe, and Asia. Our people drive Exadel’s success, and they are at the core of our values.
About Our Customer:
Top 10 North American bank
About Our Project:
The client needs to build a system for tracking deal workflows during collaborations between investors and issuers. One of the challenges is to create a Data Lake with elements of data virtualization and data governance.
Data will be used for further analytics in reporting systems and for feeding internal web applications. Web applications will be able to perform data enrichment. Initial data source processing will be done on the client side. Our team will need to set up file processing and ingestion, create data quality checks, normalize data and setup lineage, and create a data catalog.
Project Team:
When you join our team, you'll be immersed in a culture where teammates always help each other achieve better results. We believe that together we are greater and that we can find brilliant solutions by sharing ideas.
Project Tech Stack:
- AWS: S3, Glue, Lambda, EMR, RDS, Neptune
- Programming languages: Java/Scala, SQL
Requirements:
- 3+ years of working experience with data
- 1+ years of experience with Java/Scala
- Hands-on experience in implementing data lake and ingestion processes using AWS Services: S3, RDS, Glue
- Hands-on experience in setting up AWS services
- Hands-on experience in relational databases architecture and SQL
- Experience in normalizing data and setting row-based security
English level:
Intermediate+
Responsibilities:
- Communicate with the client and formalize requirements in cooperation with the business analyst
- Discuss architectural approach with the client’s architecture team and perform suggestions
- Create a Glue ingestion layer
- Setup Clue Crawler
- Create Glue Data Catalog
- Create custom data quality checks rules
- Create file processing: normalization, enrichment, and lineage using Spark
- Setting row-based security rules
- Participate in data QA automation setup
Advantages of Working with Exadel:
- You can build your expertise with our Client Engagement team, who provide assistance with existing and potential projects
- You can join any Exadel Community or create your own to communicate with like-minded colleagues
- You can participate in continuing education as a mentor or speaker. You will not only be emotionally but also financially rewarded for mentoring
- You can take part in internal and external meetups as a speaker or listener. We support you in broadening your horizons and encourage knowledge sharing for all of our employees
- You can learn English with the support of native speakers
- You can take part in cultural, sporting, charity, and entertainment events
- Working at Exadel means always upgrading your skills and proficiency, so we provide plenty of opportunities for professional development. If you’re looking for a challenge that will lead you to the next level of your career, you’ve found the right place
- We work hard to ensure honest and open relations between employees and leadership, so our offices are friendly environments
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Architecture AWS Data governance Data QA Data quality Engineering Java Lambda RDBMS Scala Security Spark SQL
Perks/benefits: Career development Team events
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open AI Engineer jobs
- Open Data Science Manager jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Senior Data Architect jobs
- Open Product Data Analyst jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open Azure Data Engineer jobs
- Open Data Product Manager jobs
- Open ETL Developer jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Snowflake-related jobs
- Open Consulting-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open NLP-related jobs
- Open Airflow-related jobs
- Open Data warehouse-related jobs
- Open LLMs-related jobs
- Open Databricks-related jobs
- Open Hadoop-related jobs