Data Engineer (Collection)

What you’ll do:

In this role, you will expand our collection capabilities by building and improving harvesters to collect data from Dark Web Forums, Paste sites, and other crucial sources. Our harvesting pipeline reads over 700,000 web sources and structured data feeds, and our real-time multilingual NLP technology turns raw text into alerts and visualizations within minutes — what you build will become part of our Security Intelligence product to help our clients stay protected. 

What you’ll bring:

Required experience and skills:

• 2+ years web scraping experience: capable of writing durable and creative web scrapers with your own custom logic on top of Python libraries like scrapy, requests, or selenium
• BA, BS, MS, or PhD in Computer Science or related discipline
• Proficiency in Python: ability to work all the way from high level architecture design down to efficient code.
• Strong communication skills and ability to collaborate

Preferred experience and skills:

• Selenium and SeleniumGrid
• Data analytics, data mining, or other data science skills
• Database experience, preferably working with Mongo databases
• Experience working with data in Information Security, Cybersecurity, or Threat Intelligence
• Experience working with bulletin boards and forums

Why should you join Recorded Future?
From over 35 nationalities, our Futurists are the perfect recipe of humility, accountability, and collaborative attitudes. Our dedication to empowering clients with elite intelligence to disrupt adversaries has earned us a 4.7-star user rating from Gartner and 8 of the top 10 Fortune 100 companies as clients.

Want more info? 
Blog & Podcast: Learn everything you want to know (and maybe some things you’d rather not know) about the world of cyber threat intelligence
Instagram & Twitter: What’s happening at Recorded Future
The Record: The Record is a cybersecurity news publication that explores the untold stories in this rapidly changing field
Timeline: History of Recorded Future

We are committed to maintaining an environment that attracts and retains talent from a diverse range of experiences, backgrounds and lifestyles.  By ensuring all feel included and respected for being unique and bringing their whole selves to work, Recorded Future is made a better place every day.

Recorded Future will not discharge, discipline or in any other manner discriminate against any employee or applicant for employment because such employee or applicant has inquired about, discussed, or disclosed the compensation of the employee or applicant or another employee or applicant.

Recorded Future is an equal opportunity and affirmative action employer and we encourage candidates from all backgrounds to apply. Recorded Future does not discriminate based on race, religion, color, national origin, gender including pregnancy, sexual orientation, gender identity, age, marital status, veteran status, disability or any other characteristic protected by law.