Senior Data Engineer (Azure DataBricks/datalake)
East Windsor, NJ, United States
Applications have closed
SpectraMedix
Health plans and their networks are moving to Value-Based Payments (VBP). The right contracting and analytics software is critical to success.Company Description
SpectraMedix:
We provide actionable insights to make our clients successful with Value Based Care Initiatives.
SpectraMedix empowers our US healthcare clients to improve quality and achieve their financial goals. Our clinical and population health intelligence solutions deliver meaningful real-time data analytics that enable reductions in preventable utilization, improve quality measure scores and maximize the clinical and financial effectiveness of investments in quality and care management improvement initiatives for progressive provider organizations, accountable care organizations (ACOs), health plans, and other risk bearing entities.
Role: Senior Data Engineer
Location: East Windsor, New Jersey
Experience: 7-12 Years
Job Description
This position will work closely with lead developers, architects and data/business analyst, analyze, implement, test and document as needed. Involves working with the client relations and project management teams insuring the SpectraMedix product is meeting the highest standards of quality and satisfying its customers.
- Create, Maintain and update Data Models
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL/Spark technologies.
- Work in data transformations and ETL using Azure technologies
- Design and implement performant data ingestion dataflows from multiple sources using Azure Databricks
- Ability to build processes that support data transformation, workload management, data structures, dependency and metadata
- Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues
- Ability to perform root cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions
- Deployment of data pipelines/ETL and Testing in UAT/Production Environment.
Qualifications
Bachelor / Masters or equivalent degree in any engineering or technical / quantitative field.
Technical Expertise:
- Strong data modelling experience
- Working knowledge on building real-time data pipelines using Apache Spark or Kafka, ETL tools, Azure Data factory, Azure Databricks etc.
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Strong analytic skills related to working with unstructured datasets
- Knowledge of Azure services such as Azure Storage, Azure Data Lake and Azure SQL Data Warehouse
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Strong project management and organizational skills.
- Experience supporting and working with cross-functional teams in a dynamic environment
- Java / python (Preferable)
- Talend (preferable)
Additional Information
Benefits:
- Competitive Salary
- Paid Time Off
- Sick Time
- Health Insurance
- Dental Insurance
- Free Gym Membership
- Monthly/Annual Company Social Events
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Azure Big Data Data Analytics Databricks Data pipelines Data warehouse Engineering ETL Kafka Pipelines Python RDBMS Spark SQL Talend Testing
Perks/benefits: Competitive pay Fitness / gym Health care Team events
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open AI Engineer jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Product Data Analyst jobs
- Open Senior Data Architect jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Principal Data Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open NLP-related jobs
- Open Data governance-related jobs
- Open LLMs-related jobs
- Open Airflow-related jobs
- Open Data warehouse-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs