Senior Data Engineer

Recorded Future collects intelligence from many sources on the Dark Web. This data collection includes terabytes of data dumps that are shared and trafficked in the underground, nefarious dark web forums, markets for stolen goods, and most notably — ransomware sites. The Ransomware squad on the Dark Web team builds the data collection and analytics that transform this mass of raw data from ransomware blogs into structured intelligence that our customers use to defend their people and networks.

The Senior Data Engineer will work on producing a constant stream of fresh, high-fidelity analytics derived from unreliable dark web and underground data sources that enable clients to track threat actors and take security actions. Your contribution will directly impact the success of the Dark Web team’s mission: to secure the world with intelligence by casting a light upon the dark web

What You’ll Do:

• Build, maintain, monitor and Collect data from hard-to-reach dark web sites and ransomware blogs.
• Own the reliability of the automatic processing of this structured raw data. Investigate and resolve data processing problems by improving our data processing code so it will automatically handle similar scenarios in the future.
• Maintain and improve our data ingestion pipeline that processes and indexes metadata from large files. This includes analytic-generation services that use NLP to extract content with the highest security value.
• Lead by example when architecting, collaborating and delivering new and improved solutions.

What You’ll Bring :

• 5 years experience with Python working in a professional software engineering environment that involves thorough code testing and review processes
• Strong problem solving capabilities and experience troubleshooting data issues within your own processes and those of your peers.
• 5 years experience maintaining production-ready, efficient data engineering pipelines and systems with extensive monitoring to accommodate complex data ingestion problems
• Very flexible and outcome-oriented. Comfortable in a work environment where urgent issues arise often, and where quickly resolving issues has immediate security value for our end customers
• A solid understanding of cybersecurity, along with an interest in analyzing dark web activities related to computer and network security.
• Experience with NLP, as well as MongoDB, ElasticSearch, and AWS cloud services is highly desirable.

The base salary range for this full-time position is $129,000-$193,500. Our salary ranges are determined by role, level, and location. The range displayed reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by state, work location and additional factors, including job-related skills, experience, and relevant education or training. This position may be eligible for incentive compensation, equity, and medical, dental, vision, life insurance and 401K. Your recruiter can share more about the specific details of the compensation and benefit package during the hiring process.