Principal Data Engineer

How you will helpAs a lead and subject matter expert on the team, you will support the enterprise data endeavors, diving in to fix issues, optimize processes and automate what you do more than once. You’ll use the best tools for the job, whether modern and revolutionary or time tested and proven, to deliver elegant, scalable solutions that meet business and technical needs.  
What you will do• Research and implement new technologies with a team of developers to execute strategies and implement solutions• Provide thought leadership, best practices on architecture and tooling, plus lead the establishment of standards for designing and implementing scalable data solutions• Solve complex problems related to the real-time discovery of large data to the scale of terabytes • Lead conversations with data product owners and business partners to ensure requirements translate into data engineering solutions, establish procedures and best practices for transforming and storing data• Lead requirements gathering around data pipeline automation improvements• Work with some of the most exciting open-source tools like Spark, Kafka, Hadoop, Docker, DataBricks and Airflow• Leverage distributed computing and serverless architecture such as AWS EMR & AWS Lambda, to develop pipelines for transforming data• Marvel at the speed with which your creation makes it into production• Produce peer reviewed quality software• Provide mentorship and guidance to data engineers• Proactively anticipating challenges to provide viable suggestions as SME and Technical lead
About you• Experienced in writing scalable applications on distributed architectures with large volumes of data• Data driven, testing and measuring as much as you can• Eager to both review peer code and have your code reviewed• Comfortable on the command line and consider it an essential tool• Confident in SQL/pyspark, you know it, write smart queries, it’s no big deal
Desired skills and experience• 10+ years of strategic experience solving data and ETL problems directly with business partners• 8+ years of large volume of data distribution with complex mappings• 8+ years of experience with Databricks, PySpark and Spark-SQL (writing, testing, debugging spark routines)• 5+ years of experience architecting, building and maintaining complex, multi-component big data systems• Working knowledge of Airflow or other orchestration and SQL code management tools• Bonus: Experience with ETL, data pipeline scheduling technologies• Bonus: Experience on distributed architectures such as microservices SOA, RESTful APIS and data integration architectures
Base salary for the role is commensurate with experience and can range between $63,000 - 250,000 + annual bonus opportunity.
About HealthVerityAt HealthVerity we are actively solving some of the greatest challenges in healthcare through innovative technology and data solutions. Our customers and partners including pharmaceutical manufacturers, payers and government organizations look to HealthVerity to partner on their  most complicated use cases, leveraging our transformative technologies and real-world data infrastructure. The HealthVerity IPGE platform, based on the foundational elements of Identity, Privacy, Governance and Exchange, enables the discovery of RWD across the broadest healthcare data ecosystem, the building of more complete and accurate patient journeys and the ability to power best-in-class analytics and applications with flexibility and ease. To learn more about the HealthVerity IPGE platform, visit www.healthverity.com.
Why you'll love working here
We are making a difference – Our technology is at the forefront of some of the biggest healthcare challenges in the world. We are one team – Our people define our culture and always will. We take time out to celebrate each other at the end of every week through company-wide shout outs, and acknowledge the value that each of us adds towards our greater mission. Come share all you have to offer.We are learners – Every team member is continually learning, no matter if we've been in a role for one year or much longer. We are committed to learning and implementing what is best for our clients, partners, and each other.
Benefits & Perks• Compensation: competitive base salary & annual bonus opportunity (for non-commissioned roles)• Benefits: comprehensive benefits with coverage on Day 1, medical, dental, vision, 401k, stock options• Flexible location: our HQ is in Philadelphia with 50% of the team distributed across 25+ states • Generous PTO: Take time off as needed, targeted at 4 weeks per year, including vacation, personal and sick time, plus paid maternity and paternity leave.• Comprehensive and individualized onboarding: mentorship program, departmental talks, and a library of resources are available beginning day 1 for each new team member to minimize the stress of starting a new job• Professional development: biweekly 1:1s, hands-on leadership that is goal-and growth-oriented for each team member, and an annual budget to support professional development pursuits
HealthVerity is currently able to employ individuals residing in the following states: AZ, CA, CT, DC, DE, FL, GA, IL, IN, MA, MD, MI, MN, MO, NC, NH, NJ, NV, NY, OH, OK, PA, RI, SC, TN, TX, UT, VA, VT, WA, WI.
HealthVerity is an equal opportunity employer devoted to inclusion in the workplace. We believe incorporating different ideas, perspectives and backgrounds make us stronger and encourages an environment where ageism, racism, sexism, ableism, homophobia, transphobia or any other form of discrimination are not tolerated. All qualified job applicants will be given consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability. At HealthVerity, we’re working towards an innovative and connected future for healthcare data and believe the future is better together. We can only do that if everyone has a seat at the table. Read our Equity Inclusion and Diversity Statement.
If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please direct your inquiries to careers@healthverity.com
Remote opportunities are not available in all areas and require team members to work from a fixed location due to tax and labor law implications - specific questions about remote positions can be discussed during the interview process with your recruiter.