Senior Big Data Engineer (AWS+Python+Hadoop)

We are currently recruiting for a Senior Big Data Engineer (AWS+Python+Hadoop) with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 3+ years to 6 years specifically in Data Engineering.
You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a large variety of exciting and challenging projects and work with our exceptional team of Technology professionals.

Technology Stack Used & Required Knowledge:

• Over 3+ years of experience on ETL Data Engineer, Python, Java, Bigdata (Hadoop, spark, hive)
• Excellent understanding of ETL cycle.
• Experience in creating and driving large-scale ETL pipelines in AWS-based environment
• Experience with integration of data from multiple data sources.
• Good knowledge and understanding of Big data engineering, ETL, Data Modeling, and Data Architecture.
• Strong software development and programming skills with a focus on data using Python/ PySpark and/or Scala for data engineering.
• Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Lambda, Athena, and Redshift will be a plus
• Understanding of data types/ handling of different data models.
• Experience with the AWS data management tools such as Data lake and Databricks or AWS Snowflake is a plus
• Understanding of descriptive and exploratory statistics, predictive modelling, evaluation metrics, decision trees, machine learning algorithms is a plus
• Good scripting and programming skills.
• Should have experience in Jira and Agile development methodologies.
• Must be open to learn and the ability to grasp new technologies quickly.
• Be motivated and passionate about technology.
• Team player with strong integrity and professionalism who can work well with others in the group.
• Self-starter with a proven ability to take initiative and strong problem-solving skills.

To keep it short, below are some key responsibilities:

• Design and develop using Big Data technologies.
• Design and develop cutting edge Big Data, and Cloud technologies to build critical, highly complex distributed systems from scratch
• Design, develop and manage complex ETL jobs and pipelines. 
• Act as a team member, consulting other teammates as your new systems transform the way this entire firm leverages data
• Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc. 
• Support and helping them learn and improve their skills, as well as innovate and iterate on best practices.
• Solve complex issues and provide guidance to the team members when needed.
• Make improvement and process recommendations that have an impact on the business

About NorthBay NorthBay, an 8+ Year Award Winning AWS Premier Consulting Partner, is an Global Analytics, DB/App Migration, App Dev and DevOps professional services company.  NorthBay's intense focus is unparalleled in the areas of Big Data (Data Lakes, Data Platforms, Enterprise Data Warehouse), Database & Application Migrations and Machine Learning/AI. NorthBay has earned the Analytics, DevOps, Mobile; Education Competency and is also a Public Sector Partner and we are also certified for AWS Well Architected Reviews and Database Freedom Migrations. What We Offer·       Consulting environment with a hyper-growth company·       Multi-cultural environment ·       An opportunity to solve complex business challenges and problems for large corporations·       International Exposure and Experience.·       Competitive salary and benefits·       Paid holidays and vacations·       Medical OPD & Health Insurance·       Fuel Allowance·       Employee Life Insurance·       GYM/Health Club Fee Reimbursement·       Mobile & Internet Allowance·       Opportunities to make a difference in a highly productive environment