DHTS IT ANALYST - Sr. Data Engineer

At Duke Health, we're driven by a commitment to compassionate care that changes the lives of patients, their loved ones, and the greater community. No matter where your talents lie, join us and discover how we can advance health together.
About Duke Health Technology Solutions
Pursue your passion for caring and innovation with Duke Heath Technology Solutions, which is dedicated to the transformation, development, and management of enterprise information technology solutions across Duke Health. By harnessing the power of innovative technologies like cloud computing and artificial intelligence — and pairing them with a forward-thinking approach — Duke Health Technology Solutions is revolutionizing the future of health care at Duke Health and beyond.

POSITION DESCRIPTION

Job Title:  IT Developer II (Data Engineer)

DHTS: Duke Health Technology Solutions (DHTS) is a robust, specialized division of Duke University Health System dedicated to the development and management of enterprise IT systems. A 2018 ‘Most Wired’ health system, Duke is nationally recognized for IT and information management as the first healthcare system to achieve the Davies Award – highest honor by the Healthcare Information and Management Systems Society (HIMSS) – for inpatient, ambulatory, and analytics health information technology capabilities. Duke was also the first to achieve Stage 7 HIMSS for Hospital EMR, Ambulatory EMR, and Analytics adoption. Our employees are among the top-skilled IT experts in the Triangle. We partner with leading scholars, clinicians, and researchers across Duke University and Duke Health to develop innovative technologies that support our mission of delivering tomorrow's healthcare today.

Primary Role:  This position reports to the DHTS-Data Partnerships, Director of Data and Analytics Platforms.  This individual will be primarily responsible for the development of data integration and delivery pipelines while also expanding the FHIR-based content stored within Duke’s Enterprise Analytics Platform.  These solutions should capitalize on technologies to improve the value of analytical data, improve the effectiveness of information stewardship, and streamline the flow of data in the organization.  Solutions should focus on using state-of-the-art data and analytics tools including traditional and near real-time data warehousing, big data, relational and document-based databases using both extract, load, transform (ELT) toolsets as well as REST APIs and FHIR.  The ideal candidate will be comfortable with data science platforms with proven experience leveraging DevOps and Automation/Orchestration tools.

Essential Tasks/Responsibilities

• Create and maintain optimal data pipeline architecture.
• Develop a data lake on Microsoft Azure using the medallion architecture leveraging a delta lake format for the silver layer.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Recommend design of analytics solutions that improves data integration, data quality, and data delivery with an eye toward re-useable components
• Articulate differences, advantages, and disadvantages between architectural solution methods
• Work with Agile team members to document and execute test plans for data loading and data validation scripts. Support the code promotion process through development and production as required by using standard CI/CD processes.
• Develop, implement, and maintain schedule/dependency logic for automated ETL processing.
• Develop monitoring, logging, and error notification processes to ensure data is updated as expected and processing metrics reported.
• Participate in the creation and maintenance of standards for coding, documentation, error handling, error notification, logging, etc.
• Accountable for conforming to established architectural, developmental, and operational standards and practices including the creation of metadata.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Evaluate and recommend database development tools
• Assist in application and database performance tuning
• Participate in system architecture design
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Share troubleshooting and maintenance duties

Education: Bachelor's degree in a related field, or four years of equivalent technical experience required

Required Experience:  We are looking for a candidate with 5+ years of experience in a Data Engineer role who should also have experience using the following software/tools:

• • Experience implementing data lakes on Azure
  • Experience with relational SQL and NoSQL databases
  • Experience with data pipeline and workflow management tools such as Azure Data Factory, Synapse Analytics
  • Experience with the FHIR model for healthcare
  • Experience with object-oriented/object-function scripting languages such as Python or Java.
  • Experience with Cloud-based analytic platforms such as Azure Synapse

Required Skills:

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets on Azure.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
• A successful history of manipulating, processing, and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Working experience with the FHIR specification; including implementation.
• Experience supporting and working with cross-functional teams in a dynamic environment.

Desired Skills:

• Knowledge of APIs, API Integration, and API Management
• Working knowledge of DevOps & Automation/Orchestration
• Knowledge of open-source software solutions and open-source as a business model
• Experience in data science using Python or R
• Technical breadth across application development, enterprise architecture, or application integration
• Understanding of Agile methodology
• Prior experience in health care related field is a plus

The information above describes the general nature and level of work assigned to this position. It is not intended to be an exhaustive list of all duties and responsibilities required of position incumbents.

Duke is an Affirmative Action/Equal Opportunity Employer committed to providing employment opportunity without regard to an individual's age, color, disability, gender, gender expression, gender identity, genetic information, national origin, race, religion, sex, sexual orientation, or veteran status.

Duke aspires to create a community built on collaboration, innovation, creativity, and belonging. Our collective success depends on the robust exchange of ideas—an exchange that is best when the rich diversity of our perspectives, backgrounds, and experiences flourishes. To achieve this exchange, it is essential that all members of the community feel secure and welcome, that the contributions of all individuals are respected, and that all voices are heard. All members of our community have a responsibility to uphold these values.

Essential Physical Job Functions: Certain jobs at Duke University and Duke University Health System may include essentialjob functions that require specific physical and/or mental abilities. Additional information and provision for requests for reasonable accommodation will be provided by each hiring department.