Senior Data Engineer (Azure DataBricks/datalake)

Company Description

SpectraMedix:
We provide actionable insights to make our clients successful with Value Based Care Initiatives.

SpectraMedix empowers our US healthcare clients to improve quality and achieve their financial goals. Our clinical and population health intelligence solutions deliver meaningful real-time data analytics that enable reductions in preventable utilization, improve quality measure scores and maximize the clinical and financial effectiveness of investments in quality and care management improvement initiatives for progressive provider organizations, accountable care organizations (ACOs), health plans, and other risk bearing entities.

Role: Senior Data Engineer 
Location: East Windsor, New Jersey 
Experience: 7-12 Years 

Job Description

This position will work closely with lead developers, architects and data/business analyst, analyze, implement, test and document as needed. Involves working with the client relations and project management teams insuring the SpectraMedix product is meeting the highest standards of quality and satisfying its customers.

• Create, Maintain and update Data Models
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL/Spark technologies.
• Work in data transformations and ETL using Azure technologies
• Design and implement performant data ingestion dataflows from multiple sources using Azure Databricks
• Ability to build processes that support data transformation, workload management, data structures, dependency and metadata
• Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues
• Ability to perform root cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions
• Deployment of data pipelines/ETL and Testing in UAT/Production Environment.

Qualifications

Bachelor / Masters or equivalent degree in any engineering or technical / quantitative field.

Technical Expertise:

• Strong data modelling experience
• Working knowledge on building real-time data pipelines using Apache Spark or Kafka, ETL tools, Azure Data factory, Azure Databricks etc.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Strong analytic skills related to working with unstructured datasets
• Knowledge of Azure services such as Azure Storage, Azure Data Lake and Azure SQL Data Warehouse
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment
• Java / python (Preferable)
• Talend (preferable)

Additional Information

Benefits:

• Competitive Salary
• Paid Time Off
• Sick Time
• Health Insurance
• Dental Insurance
• Free Gym Membership
• Monthly/Annual Company Social Events