Senior Data Engineer II
New York
Applications have closed
DoubleVerify
DoubleVerify is driven by a mission – to make the digital advertising ecosystem stronger, safer and more secure.Company: DoubleVerify
Role: Senior Data Engineer II (P4) - Social Integrations
Type: Full Time
Reports To: Team Lead, Social Integration
Location: New York, NY
Who we are
DoubleVerify is the leading independent provider of marketing measurement software, data, and analytics that authenticates the quality and effectiveness of digital media for the world's largest brands and media platforms. DV provides media transparency and accountability to deliver the highest level of impression quality for maximum advertising performance. Since 2008, DV has helped hundreds of Fortune 500 companies gain the most from their media spending by delivering best-in-class solutions across the digital ecosystem, helping to build a better industry. Learn more at www.doubleverify.com.
The Role
As a Senior Data Engineer II in Social Integrations, you own new initiatives, and design and build world-class data systems to ingest and analyze billions of records per day from the world’s biggest social platforms, like Facebook, Youtube, Snap, and more. You build high-volume, high-availability, low-latency APIs to support live traffic metrics collection. You develop systems that analyze and categorize social content to power tools that allow advertisers to understand and control where their ads run. You use state-of-the-art technologies, frameworks, and strategies to address complex challenges with Big-Data processing and analytics to achieve the above objectives.
What you’ll do
- Write solid code with a focus on high performance for services supporting high-throughput and low-latency
- Architect, design, and build big-data-processing platforms, handling tens of TBs/day, serve thousands of clients and support advanced analytic workloads
- Provide meaningful and relevant feedback to junior developers and stay up-to-date with technology changes
- Explore the technological landscape for new ways of producing, processing, and analyzing data, in order to gain insights on our users and product features
- Design, develop, and test data-driven products, features, and APIs that scale
- Continuously improve the quality of deliverables and SDLC processes
- Operate production environments, investigate issues, assess their impact, and come up with feasible solutions
- Understand business needs and work with product owners to establish priorities
- Translate between business / product requirements and technical details
- Work in multi-functional agile teams with end-to-end responsibility for product development and delivery
Who you are
- 5+ years of programming experience in coding, object-oriented design, and/or functional programming including Python, Scala, or related language
- Lead by example - design, develop and deliver quality solutions
- Love what you do and are passionate about crafting clean code
- Deep understanding of distributed system technologies, standards, and protocols, and have 2+ years of experience working in distributed systems like Airflow, BigQuery, Spark Streaming, Kafka ecosystem ( Kafka Connect, Kafka Streams, or Kinesis), and building data pipelines at scale
- Hands-on experience building low-latency, high-throughput APIs, and comfortable using external APIs from platforms
- Expertise in relational database concepts, data modeling and crafting complex SQL queries
- Cares about agile software processes, data-driven development, software reliability, and responsible experimentation
- Genuine desire to automate decision-making, processes, and workflows
- Experience working with process orchestration tools such as Luigi/Airflow
- Experience with DevOps domain - working with build servers, docker, and containers clusters (kubernetes)
- Experience mentoring and growing a diverse team of talented data engineers
- B.S./M.S. in Computer Science or a related field
- Excellent communication skills and a team player
- Experience with the following technologies is a plus:
- Columnar data stores
- Cloud environment, Google Cloud Platform
- Ad-serving technologies and standards
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow APIs BigQuery Computer Science Data pipelines DevOps Distributed Systems Docker GCP Google Cloud Kafka Kinesis Kubernetes Pipelines Python Scala SDLC Spark SQL Streaming
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Data Manager jobs
- Open Sr Data Engineer jobs
- Open Power BI Developer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Data Scientist II jobs
- Open Product Data Analyst jobs
- Open Senior Data Architect jobs
- Open Sr. Data Scientist jobs
- Open Business Data Analyst jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Data Product Manager jobs
- Open Junior Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open GCP-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Snowflake-related jobs
- Open Consulting-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Hadoop-related jobs
- Open LLMs-related jobs
- Open Databricks-related jobs
- Open Data warehouse-related jobs