Senior Data Engineer

We are seeking a talented and experienced Senior Data Engineer to play a key role in the development of our cutting-edge data collection, processing, and analysis system, which is based on a Python/Django/Postgres stack. Working closely with the Lead System Architect and Data Processing Engineer, you will be responsible for implementing and optimizing data pipelines, processing components, and vulnerability review interfaces that will catalog Python, Java, and JavaScript vulnerabilities.

Responsibilities:

- Collaborate with the Lead System Architect to design and implement data ingestion, processing, and review components

- Develop and optimize data pipelines, ETL processes, and data cleaning mechanisms

- Contribute to the development of LLM analysis components

- Experiment with and discover novel ways to detect potential vulnerabilities in packages or software

- Ensure the implemented components align with the overall system architecture and meet performance, scalability, and maintainability requirements

- Participate in code reviews and contribute to the development of best practices and coding standards

- Assist in troubleshooting and resolving technical issues

- Embrace a fast-paced, iterative development approach, quickly delivering working solutions and continuously improving them based on feedback

Impact:

As a Senior Data Engineer, you will play a crucial role in building a groundbreaking data processing system that will result in an industry-leading dataset, protecting and securing the Python, Java, and JavaScript ecosystems. Your work will have a massive impact on the cybersecurity landscape, empowering organizations worldwide to safeguard their software supply chains and mitigate vulnerabilities. Be part of a team that is at the forefront of innovation, leveraging cutting-edge AI technologies to revolutionize the way we approach cybersecurity.

If you are excited about having the opportunity to make a significant impact in the cybersecurity domain and build a world-class data processing system, we want to hear from you! Join our dynamic and fast-paced startup, where you'll have the chance to work with cutting-edge technologies, shape the future of software supply chain security, and deliver impactful results through iterative deployments.

Requirements

- 10+ years of overall technical experience

- 7+ years of experience in data engineering and processing

- Strong skills in data manipulation, transformation, and analysis

- Proficiency in relevant programming languages and tools (e.g., Python, Java, SQL)

- Experience with big data technologies and cloud platforms (AWS or Google Cloud preferred)

- Familiarity with event-driven architectures and data pipeline design

- Excellent problem-solving and communication skills

- Experience using AI systems like GPT, Claude, and Copilot in daily work

- Passionate about leveraging AI tools to drive innovation and efficiency

- Thrives in a fast-paced startup environment, comfortable with rapid iterations and adaptable to changing requirements

- Resourceful and creative problem-solver, able to deliver results with limited resources and tight deadlines