Principal Data Engineer

About Zignal Labs
Zignal Labs’ real-time intelligence technology helps the world’s largest organizations protect their people, places, and position. Analyzing billions of data points in real time,  Zignal's AI-powered platform accelerates mission-critical decision making by empowering leaders with contextual situational awareness of the information environment.
Fully remote, with Silicon Valley roots and team members in over 20 states, Zignal serves customers around the world. Learn more at zignallabs.com.
As the Principal Data Engineer on the Platform team at Zignal Labs, you will get to use your Scala and Java experience to build a best-in-class distributed data and analytics infrastructure by leveraging open source technologies such as Apache Spark, Apache Storm, and Elasticsearch.  We use social media, news, blogs and other media sources to empower our users with key insights based on real-time analysis.

In this role, you will have the opportunity to:

• Solve complex real-time data collection & analysis problems with cutting edge technical solutions
• Iterate on our high performance and scalable platform for massive data collection, real-time analytics, NLP, machine learning, and backend data services
• Build high performance, scalable, real-time, server-side technologies
• Write scalable code with extensive test coverage, working in a professional software engineering environment with source control, dev/stage/production release cycles, continuous integration, and deployment
• Work closely with product management, design, quality assurance and operations teams to understand our customers’ needs and effectively translate them to technical specifications
• Lead projects from translating product requirements into architecture to production

Tech Stack:

• Scala, Java, Python
• Apache Spark, Spark Streaming, Databricks/Delta Lake, Apache Storm, Elasticsearch, Apache Nifi
• Kafka, MongoDB, Redis
• AWS

In order to be successful in this role, you will need:

• Bachelor's degree (or higher) in Computer Science, Engineering, or similar and/or relevant work experience
• Experience providing technical leadership at the enterprise level for the design of information technology systems
• Crafted and implemented operational data stores, as well as data lakes in production environments
• Ability to analyze, diagnose and resolve complex architectural problems using industry standard engineering principles
• Design and build data ingestion pipelines and ETL processing, including stream processing, while factoring in performance and cost
• Identify and solve issues concerning data management to improve data quality
• Clean, prepare and optimize data for ingestion and consumption
• Experience solving performance problems with Lucene based search solutions like Elasticsearch or Solr
• 9+ years experience in server-side/back-end full cycle product development in a production environment
• 4+ years developing with Apache Spark, including Structured Streaming.   Experience with Databricks is a big plus
• Knowledge of Scala or Java with exposure to or interest in Scala
• Leads and mentors other team members
• Provides partners with coaching and feedback in order to build effective teams
• Provides effective support to cross-functional teams

Why join Zignal Labs?- Competitive salary based on the work you do- Flexible time off – work with your manager to take the time you need- Excellent medical, dental, and vision coverage- Paid parental leave plan- Professional development and growth programs - A tight knit, collaborative, and transparent environment to help you succeed
Zignal Labs is proud to be an Equal Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.
Applicants must be authorized to work in the United States for any employer. No sponsorship is available for this position.