Lead Data Engineer-Delivery

Company Description

We help the world see new possibilities and inspire change for better tomorrows. Our analytic solutions bridge content, data, and analytics to help business, people, and society become stronger, more resilient, and sustainable.

Job Description

In this role you will be responsible for managing the data engineering team, expanding and optimizing our data pipeline architecture, as well as enabling delivery of new data engineering products at increasing scale. The Lead Data Engineer will support Product Delivery, Engineering, Analytics, and all of our data scientists on various initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. 

ESSENTIAL FUNCTIONS: 

• Help us grow the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, AWS “big data” technologies, and Snowflake. 
• Assemble large, complex data sets that meet functional / non-functional business requirements. 
• Enhance, create and maintain toolsets to help manage our enterprise data pipelines, 
• Ensure your teams and your own code meets quality standards and architectural requirements 
• Participate in code reviews 
• Lead, mentor, and manage other data engineers 
• Design and architect data engineering solutions 
• Work with product leads and other technology leads to break down technical and non-technical requirements into actionable work items (stories, acceptance criteria, and other agile artifacts). 
• Help keep our data separated and secure across in cloud environments (AWS, Snowflake, etc) 

#LI-MC2

Qualifications

REQUIRED QUALIFICATIONS 

• Intermediate/Advanced knowledge of SQL and NoSQL databases, and an understanding of key concepts (Foreign Keys, Indices, stored procedures, etc) 
• Experience using Snowflake data warehouses, stored procedures, roles, and integration of Snowflake to other systems 
• Intermediate/Advanced knowledge of Python 
• Intermediate/Advanced Experience with DevOps/architecture as code, focused on serverless architectures, Docker, Kubernetes, containerization and other autoscaling solutions for “big data” processing, including cluster management and Apache Airflow deployment/management. 
• Experience with and knowledge of large batch data processing and ETL/ELT systems 
• Intermediate or better understanding of bash scripting, and the basics of Linux operating systems. 
• Experience with source control and CI/CD process and tools, such as Jenkins, Gitlab and related scripting (terraform, serverless framework, SAM, etc) 
• Experience seeing the full end to end SDLC of a system and processes to production. 
• Experience with Serverless architectures and data solutions 
• Experience working in multi-engineer, team driven, shared environments and workflows, and feature based development best practices 
• Experience working under agile/scrum or kanban and the major pieces of those frameworks (epics, stories, sprint planning and retros) 
• 5+ years of experience developing production ready data engineering solutions, with  experience in AWS and Snowflake or other cloud-based architectures (GDP, Azure, etc) 
• 1-2 years of people management and/or technical architecture and development leadership. 
• Experience building working relationships with Product leadership, other technology teams, IT and DevOps teams 

BONUS QUALIFICATIONS 

• Intermediate/Advanced knowledge of and experience with Big Data technologies, preferably PySpark/ Spark, or Pandas, Glue, EMR 
• Intermediate knowledge of Javascript 
• Experience with stream-processing systems 
• Big Data Testing best practices 
• Experience with Test Driven and Behavior Driven Development  (TDD and BDD) 
• Understanding of Microservice architecture  
• Ability to identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. 

Additional Information

Infutor Data Solutions is the expert in Consumer Identity Management and Identity Resolution, offering award-winning solutions to a variety of brands and martech firms. These world-class data solutions enable businesses to most accurately identify key consumer information, powering them to make critical, strategic business decisions. We do this through access to the most comprehensive and accurate data, provided through powerful and flexible technology, giving businesses extensive insight into the core of their business. Infutor's data solutions drive essential business functions in identity verification, fraud and risk, marketing, analytics, multi-media, and compliance applications.

Our company of rapidly-growing data, technology and analytics leaders is looking for a to join our team that has recently been recognized as:

·        Crain’s Best Places to Work Company

·        Crain’s Best-Paying Company

·        BuiltIn Best Places to Work

·        Inc. 5000 Fastest Growing Companies in the US

·        AdWeek Top 20 Fastest Growing Solution Providers

·        Martech Breakthrough Award Winner for Best Contact Database Solution

Verisk (Nasdaq:VRSK) provides predictive analytics and decision-support solutions to customers in the insurance, energy and specialized markets, and financial services industries. More than 70 percent of the FORTUNE 100 relies on the company’s advanced technologies to manage risks, make better decisions and improve operating efficiency. Celebrating its 50th anniversary, the company continues to make the world better, safer and stronger, and fosters an inclusive and diverse culture where all team members feel they belong.

With more than 100 offices in nearly 35 our global team uses AI, machine learning, automation, and other emerging technologies to collect and analyze billions of records. We provide advanced decision-support to prevent credit, lending, and cyber risks. In addition, we monitor and advise companies on complex global matters such as climate change, catastrophes, and geopolitical issues.

But why we do our work is what sets us apart. It stems from a commitment to making the world better, safer and stronger.

It’s the reason Verisk is part of the UN Global Compact sustainability initiative. It’s why we made a commitment to balancing 100 percent of our carbon emissions. It’s the aim of our “returnship” program for experienced professionals rejoining the workforce after time away. And, it’s what drives our annual Innovation Day, where we identify our next first-to-market innovations to solve our customers’ problems.

At its core, Verisk uses data to minimize risk and maximize value. But far bigger, is why we do what we do.

At Verisk you can build an exciting career with meaningful work; create positive and lasting impact on business; and find the support, coaching, and training you need to advance your career. We have received the Great Place to Work® Certification for the fifth consecutive year. We’ve been recognized by Forbes as a World’s Best Employer and a Best Employer for Women, testaments to our culture of engagement and the value we place on an inclusive and diverse workforce. Verisk’s Statement on Racial Equity and Diversity supports our commitment to these values and affecting positive and lasting change in the communities where we live and work.

Verisk Analytics is an equal opportunity employer.

All members of the Verisk Analytics family of companies are equal opportunity employers. We consider all qualified applicants for employment without regard to race, religion, color, national origin, citizenship, sex, gender identity and/or expression, sexual orientation, veteran's status, age or disability.

http://www.verisk.com/careers.html

Unsolicited resumes sent to Verisk, including unsolicited resumes sent to a Verisk business mailing address, fax machine or email address, or directly to Verisk employees, will be considered Verisk property. Verisk will NOT pay a fee for any placement resulting from the receipt of an unsolicited resume.

Consumer Privacy Notice