Big Data Engineer

Lahore, Punjab

NorthBay Solutions

Your Unwavering Compass for Everything AWS Find Your True North with NorthBay, an AWS Premier Service Provider Get access to all of our AWS best practices and insights.     Subscribe Now Navigating the Cloud is was Complex Navigating...

View all jobs at NorthBay Solutions

Position Overview:
NorthBay is looking for a Big Data Engineers with 2 to 3+ years of experience to join our team. The ideal candidate will have a proven track record in data engineering, with a focus on modern cloud-based solutions and a strong proficiency in SQL. This role requires expertise in designing, implementing, and maintaining data pipelines for both batch and streaming data processing, as well as the ability to troubleshoot data issues effectively. The Data Engineer will play a key role in supporting our data infrastructure and driving innovation in data processing and analytics.

Location: Lahore, Karachi, Islamabad
Work Scope: Hybrid

Key Responsibilities:

  • Design, implement, and maintain ETL/ELT processes for batch and streaming data processing.
  • Develop and optimize data pipelines using modern cloud data warehousing and data lake solutions such as Snowflake, BigQuery, Redshift, and Azure Synapse.
  • Investigate and troubleshoot data issues, providing short-term fixes and proposing long-term solutions to prevent recurrence.
  • Utilize AWS services including S3, DMS, Glue, and Athena to build scalable and reliable data solutions.
  • Collaborate with cross-functional teams to understand data requirements and develop solutions that meet business needs.
  • Employ data transformation tools such as dbt to efficiently transform and model data for analytics and reporting purposes.

Mandatory Requirements:

  • Bachelor's degree in Computer Science, Engineering, or related field.
  • 3+ years of experience in data engineering.
  • Proficiency in SQL.
  • Experience with modern cloud data warehousing and data lake solutions.
  • Excellent ability to investigate and troubleshoot data issues, providing fixes and proposing both short and long-term solutions.
  • Knowledge of AWS services (like S3, DMS, Glue, Athena, etc.).
  • Familiarity with dbt or other data transformation tools.

Nice to Have:

  • Proficiency in Python programming language.
  • Familiarity with infrastructure as code tools such as Terraform and Terragrunt.
  • Experience with orchestration tools like Dagster, Airflow, and AWS Step Functions.
  • Knowledge of pub-sub, queuing, and streaming frameworks such as AWS Kinesis, Kafka, SQS, and SNS.
  • Understanding of CI/CD pipelines and automation practices.

* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰

Job stats:  2  0  0

Tags: Airflow Athena AWS Azure Big Data BigQuery CI/CD Computer Science Dagster Data pipelines Data Warehousing dbt ELT Engineering ETL Kafka Kinesis Pipelines Python Redshift Snowflake SQL Step Functions Streaming Terraform

Region: Asia/Pacific
Country: Pakistan

More jobs like this

Explore more AI, ML, Data Science career opportunities

Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.