Data Engineer
Toronto
Our Culture: We believe that people matter most. And our people set us apart. We live and breathe technology every day. This is the fuel of our innovation. But we know that the more significant the technology becomes, the more human we have to be. That's why we value what really matters: our people. We have a flexible, fun and casual environment at work. CI&T is recognized as a Great Place to Work.
What will you be doing? You are going to be responsible for:- Design, develop and maintain scalable data pipelines, and build out new API integrations for data transfer.- Perform data analysis required to troubleshoot data-related issues and assist in the resolution of data issues.- Collaborate with Analysts to understand upstream data assets- Own big data enrichment pipelines using AWS technologies like Glue, Pyspark and MWAA with data hosted in S3, Snowflake, and PostgreSQL- Develop locally, empowered by Docker and tools like VSCode, and DBeaver, Jupyter Notebook, GLUE interactive sessions.- Work with QA Engineers to create & maintain tests for CI/CD gates and ETL validation- Contribute beyond the data layer by developing application layer code in Python- Build resilient CI/CD alongside DevOps using GitHub Actions and Terraform- Consult with Architects and other data engineers to define and follow best practices
What do you bring to the role? We are looking for someone with the following attributes: - BS or MS degree in Computer Science or a related technical field- 5+ years of extensive ETL development experience using Pyspark/Glue on AWS- 5+ years of experience in CSV, JSON, and Parquet file formats.- 5+ years of experience in S3, Athena, RDS, Glue catalog, Cloudformation/Terraform- Experience with querying nested/JSON data stored in parquet files or tables.- Strong understanding of ETL/Data-pipelines/BigData architecture- Strong Database/SQL experience in any RDBMS
Nice to have:- Experience in schema design, data ingestion experience on Snowflake (or equivalent MPP)- Experience in orchestrating data processing jobs using Step Function/Glue workflow/Apache Airflow (MWAA)- Experience in data analysis using Excel formulas, Vlookup, pivot, slicers
Why join us? ● We offer incredible benefits ○ Competitive Salary ○ Generous paid vacation days ○ Unlimited sick time ○ 100% paid health & dental benefits starting day one ○ Annual profit-sharing distribution ○ Retirement match ○ Paid parental leave ○ And much much more…
● We believe in building your career ○ Through our amazing Career development program that includes mentorship, career guidance and access to CI&T University so you can always be learning.
#LI-TP2#MidSenior#LI-RemoteCI&T is an equal opportunity employer. We celebrate and appreciate the diversity of our CI&Ters’ identities and lived experiences. We are committed to building, promoting, and retaining a diverse, inclusive, and equitable company and culture focused on creating a better tomorrow.
At CI&T, we recognize that innovation and transformation only happen in diverse, inclusive, and safe work environments. Our teams are most impactful when people from all backgrounds and experiences collaborate to share, create, and hear ideas. We strongly encourage Black, Indigenous, and People of Color (BIPOC), immigrants, candidates with a disability, women, and LGBTQIA+ candidates to apply to our vacancies.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow APIs Architecture Athena AWS Big Data CI/CD Computer Science CSV Data analysis Data pipelines DevOps Docker Engineering ETL Excel GitHub JSON Jupyter MPP Parquet Pipelines PostgreSQL PySpark Python RDBMS Research Snowflake SQL Terraform
Perks/benefits: Career development Competitive pay Flex vacation Health care Parental leave Unlimited paid time off
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Sr Data Engineer jobs
- Open Data Manager jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Power BI Developer jobs
- Open Junior Data Scientist jobs
- Open Business Intelligence Developer jobs
- Open Senior Data Architect jobs
- Open Data Scientist II jobs
- Open Product Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Business Data Analyst jobs
- Open Manager, Data Engineering jobs
- Open Big Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Data Analyst Intern jobs
- Open Data Product Manager jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open ETL Developer jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Java-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Hadoop-related jobs
- Open LLMs-related jobs
- Open Databricks-related jobs
- Open DevOps-related jobs