Senior Data Engineer
San Francisco, California, United States
Senior Data Engineer
6sense is a Predictive Intelligence Engine that is reimagining how B2B companies do sales and marketing. It works with big data at scale, advanced machine learning and predictive modeling to find buyers and predict what they will purchase, when and how much.
6sense helps B2B marketing and sales organizations fully understand the complex ABM buyer journey. By combining intent signals from every channel with the industry’s most advanced AI predictive capabilities, it is finally possible to predict account demand and optimize demand generation in an ABM world. Equipped with the power of AI and the 6sense Demand Platform™, marketing and sales professionals can uncover, prioritize and engage buyers to drive more revenue.
6sense is seeking a Senior Data Engineer to become part of a team developing, deploying, and scaling ETL solutions for our big data platform. The candidate will be part of the team responsible for data that is crucial to prediction and targeting.
A Data Engineer at 6sense will have the opportunity to
- Apply distributed computing and map-reduce knowledge to an environment with multiple subsystems that undergo different load patterns throughout the day
- Intellectually contribute to the software, data, process, and growth of team members
- Work on scaling issues to help with 10x our capability over the coming year
- Write custom UDFs, UDAFs, UDTFs to help simplify complex operations
- Contribute to testing tools and processes
- Design and implement tools to validate data quality as well as automate and semi-automate the feedback collection process
- Uphold data security and privacy
- Perform analyses to derive insights across multiple large data sets
- Debug any issues that arise from data pipelines.
Required qualifications and must have skills
- BE/BTech/BS or equivalent
- 2+ years work experience showing growth as a Data Engineer or Software Engineer.
- 1+ years of hands on coding experience in Java (ready to code in week 1)
- 1+ years of hands on coding experience in Python
- Experience in writing complex, optimized SQL queries joining multiple large data sets and an understanding of joins, partitioning, clustering, and skews.
- Experience with Big Data Platforms like Hadoop / Hive / Spark / Presto
- Comfortable with Unix / Linux command line
- Understanding of the map-reduce paradigm
- Analytical and problem-solving skills
- Database principles, practices, structures, and theories
Nice to have Skills
- Experience with optimizing existing queries by rephrasing the SQL
- Experience with optimizing existing queries via Hive/Hadoop settings
- Experience with writing Hive / Presto UDFs in Java
- Used Key Value stores or noSQL databases
- Good understanding of docker and container platforms like Mesos and Kubernetes
- Security-first architecture approach
- Application benchmarking and optimization
- Multi-threading and concurrency
- Job schedulers
- You can work independently as well as part of a team
- You take ownership of projects and drive them to conclusion
- You’re a good communicator and are capable of not just doing the work, but teaching others and explaining the “why” behind complicated technical decisions
- You aren’t afraid to roll up your sleeves: This role will evolve over time, and we’ll want you to evolve with it!