Senior Data Engineer (Databricks, AWS) - CANADA/REMOTE
Remote - Canada
Applications have closed
Wavicle Data Solutions
Top-rated data analytics firm offering data management consulting, data integration services, and more.Wavicle Data Solutions leverages Cloud, Data & Analytics technologies to deliver complex business & digital transformation solutions to our clients. As a Minority Business Enterprise (MBE) with a 40%+ women workforce, Wavicle fosters a diverse & equitable environment where innovative professionals come together as a team and enable our clients to realize their goals in their transformation journey. Our team members collaborate by infusing their creative problem solving skills, agile working & tech know-how to drive value for our clients.
At Wavicle, a Top Workplace award winner, you’ll find a challenging and rewarding work environment where our 350+ team members based in US, India Canada work from 42 cities in a remote/hybrid, digitally connected way. We offer a competitive benefits package that includes: healthcare, retirement, life insurance, short/long-term disability, unlimited paid time off, short-term incentive plans (annual bonus) and long-term incentive plans.
Watch here to learn Why Wavicle
We are looking for a Senior Data Engineer who will be responsible for developing event subscription and streaming services in the AWS cloud platform using Databricks.
What You Will Get To Do:
- Create the conceptual, logical and physical data models.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of sources like API, Blob Storage, and no SQL Database.
- Design, develop, test, deploy, maintain and improve data integration pipeline.
- Develop pipeline objects using Databricks / Apache Spark / Pyspark / Python.
- Design and develop data pipeline architectures using Big Data, Spark and related AWS Services.
- Load and performance test data pipelines built using the above-mentioned technologies.
- Communicate effectively with client leadership and business stakeholders.
- Participate in proposal and/or SOW development.
- Lead and/or mentor a small team of data engineers.
Requirements
- Strong hands-on experience implementing big-data solutions in the AWS ecosystem (Lambda, S3, EC2, etc.) is required.
- 2+ years of professional work experience on data pipeline implementation using Databricks is required.
- 3+ years of hands-on programming experience using Python in a cloud environment (AWS preferred) is required.
- 2+ years of professional work experience using real-time streaming systems Event Grid and Event topics is highly preferred.
- Expert level knowledge of using SQL to write complex, highly-optimized queries across large volumes of data is required.
- Experience building conceptual, logical and/or physical database designs using tools such as ErWin, Visio or Enterprise Architect.
- 2+ years of working with one or more databases like Snowflake, Redshift, Synapse, Oracle, SQL Server, Teradata, Netezza, Hadoop, Mongo DB or Cassandra is required.
- Knowledge or experience in architectural best practices in building data lakes is required.
- Strong problem solving and troubleshooting skills with the ability to exercise mature judgement.
- Ability to work independently and provide guidance to junior data engineers.
- Ability to build and maintain strong customer relationships.
- Bachelor or Master’s degree in Computer Science, Engineering, Information Systems or relevant degree is required.
- Candidates can live anywhere in Canada (Ontario, Alberta or Quebec providences) and work remotely; however, must be able to travel up to 25% to the client location (U.S. or Canada) to foster and build strong business relationships.
Equal Opportunity Employer
Wavicle is an Equal Opportunity Employer and committed to creating an inclusive environment for all employees. We welcome and encourage diversity in the workplace regardless of race, color, religion, national origin, gender, pregnancy, sexual orientation, gender identity, age, physical or mental disability, genetic information or veteran status.
Benefits
- Health Care Plan (Medical, Dental & Vision)
- Life Insurance (Basic, Voluntary & AD&D)
- Unlimited Paid Time Off (Vacation, Sick & Public Holidays)
- Short Term & Long Term Disability
- Training & Development
- Work From Home
- College Tuition Benefit
- Bonus Program
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile APIs AWS Big Data Cassandra Computer Science Databricks Data pipelines EC2 Engineering Hadoop Lambda Oracle Pipelines PySpark Python Redshift Snowflake Spark SQL Streaming Teradata
Perks/benefits: Health care Insurance Salary bonus Unlimited paid time off
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Lead Data Analyst jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Principal Data Engineer jobs
- Open Sr Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Product Data Analyst jobs
- Open Senior Data Architect jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Principal Data Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open LLMs-related jobs
- Open Airflow-related jobs
- Open Data warehouse-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs