Big Data Developer
Westminster, CO, United States
Love cutting-edge tech? We do too.
At Epsilon, we do more than collect and store data. We help some of the world’s biggest brands discover real opportunities inside the data types, delimiters and decimals.
This Big Data Engineer position will focus on designing, developing, and supporting our Hadoop data solutions in Spark and Python (PySpark) while working with other components of the Hadoop ecosystem such as HDFS, Hive, Hue, Impala, Zeppelin, Jupyter. A successful candidate will work closely with business and portfolio leads to understand requirements then design and build innovative data solutions.
What you’ll do:
- Design and development centered around PySpark, Python and Hadoop Framework.
- Working with gigabytes/terabytes of data and must understand the challenges of transforming and enriching such large datasets.
- Provide effective solutions to address the business problems – strategic and tactical.
- Collaboration with team members, project managers, business analysts and QA teams in conceptualizing, estimating and developing new solutions and enhancements.
- Work closely with the stake holders to define and refine the big data platform to achieve sales, product, and strategic objectives.
- Collaborate with other technology teams and architects to define and develop cross-function technology stack interactions.
- Read, extract, transform, stage and load (ETL) data to multiple targets, including Hadoop and Oracle.
- Ingest and streamline incoming files of various layouts/formats as part of Source Prep process
- Develop scripts around Hadoop framework to automate processes and existing flows.
- Modify existing programming/code for new requirements.
- Estimate work, and track progress through SDLC with JIRA/Confluence
- Unit testing and debugging. Perform root cause analysis (RCA) for any failed processes.
- Convert business requirements into technical design specifications and execute on them.
- Participate in code reviews and keep applications/code base in sync with version control (GIT/Bitbucket)..
- Good analytical thinking and problem-solving skills.
- Ability to diagnose and troubleshoot problems quickly.
- Motivated to learn new technologies, applications, and domains.
- Possess appetite for learning through exploration and reverse engineering.
- Strong time management skills.
- Ability to take full ownership of tasks and projects.
- Possess Can-Do attitude to overcome any kind of challenges.
What you’ll bring:
- Bachelor’s degree in computer science, engineering, mathematics, related technical discipline or equivalent experience is required
- 3+ years of experience as a Big Data Engineer or in a similar role where you’ve performed analysis, design and implementation with Hadoop distributed frameworks.
- These include Python & Spark (SparkSQL, PySpark), HDFS, Hive, Impala, Hue, Cloudera Hadoop, Zeppelin, Jupyter, and other relevant skills.
- Extensive experience handling large volumes of data (measured in Terabytes/Billions of Transactions)
- Proficient knowledge of SQL with any RDBMS
- Familiarity with RDD and Data Frames within Spark
- Working knowledge of data analytics
- Troubleshooting and complex problem-solving skills
- Knowledge of Oracle databases and PL/SQL
- Working knowledge of Linux/Unix environments and comfort with Unix Shell scripts (ksh, bash)
- Basic Hadoop administration knowledge
- DevOps Knowledge is an advantage
- Ability to work within deadlines and effectively prioritize and execute on tasks
- Strong communication skills (verbal and written) with ability to communicate across teams, internal and external at all levels
- Effective communication, self-motivation, and ability to work independently while remaining fully aligned within a distributed team environment.
- Working knowledge of Oracle databases and PL/SQL.
- Hadoop Admin & Dev-Ops.
- ETL Skills (Familiarity with Talend or other ETL tools a plus)
- Any of the following certifications:
- CCA Spark and Hadoop Developer.
- MapR Certified Spark Developer (MCSD).
- MapR Certified Hadoop Developer (MCHD).
- HDP Certified Apache Spark Developer.
- HDP Certified Developer.
Salary Range: $85,000 - $120,000
When you’re one of us, you get to run with the best. For decades, we’ve been helping marketers from the world’s top brands personalize experiences for millions of people with our cutting-edge technology, solutions and services. Epsilon’s best-in-class identity gives brands a clear, privacy-safe view of their customers, which they can use across our suite of digital media, messaging and loyalty solutions. We process 400+ billion consumer actions each day and hold many patents of proprietary technology, including real-time modeling languages and consumer privacy advancements. Thanks to the work of every employee, Epsilon has been consistently recognized as industry-leading by Forrester, Adweek and the MRC. Positioned at the core of Publicis Groupe, Epsilon is a global company with more than 8,000 employees around the world. Check out a few of these resources to learn more about what makes Epsilon so EPIC:
- Culture: https://www.epsilon.com/us/about-us/our-culture-epsilon
- DE&I: https://www.epsilon.com/us/about-us/diversity-equity-inclusion
- CSR: https://www.epsilon.com/us/about-us/corporate-social-responsibility
- Life at Epsilon: https://www.epsilon.com/us/about-us/epic-blog
Great People Deserve Great Benefits
We know that we have some of the brightest and most talented associates in the world, and we believe in rewarding them accordingly. If you work here, expect competitive pay, comprehensive health coverage, and endless opportunities to advance your career.
Epsilon is an Equal Opportunity Employer. Epsilon’s policy is not to discriminate against any applicant or employee based on actual or perceived race, age, sex or gender (including pregnancy), marital status, national origin, ancestry, citizenship status, mental or physical disability, religion, creed, color, sexual orientation, gender identity or expression (including transgender status), veteran status, genetic information, or any other characteristic protected by applicable federal, state or local law. Epsilon also prohibits harassment of applicants and employees based on any of these protected categories.
Epsilon will provide accommodations to applicants needing accommodations to complete the application process.
Applicants with criminal histories are welcome to apply.
More jobs like this
Dayton, OH-Customer Site Dayton, OH-Customer Site Full TimeSenior Senior-levelUSD 131K - 201K * USD 131K+ *
Machine Learning EngineerComputer Science Computer Vision Deep Learning Engineering Machine Learning NLP Open Source +9
Career development Competitive pay Flex hours Flex vacation Health care +3
Remote - USA Remote - USA Full TimeSenior Senior-levelUSD 115K - 180K * USD 115K+ *
Lead Data Engineer (Remote Positions Available)Agile AWS Business Intelligence Data analysis Data management Data pipelines Data warehouse +17
401(k) matching Career development Gear Health care Home office stipend +4
Mountain View, USA Mountain View, USA Full TimeSenior Senior-levelUSD 192K - 356K USD 192K+
Senior Staff Engineer – Global Operation Data Science (GODS)Agile AWS Computer Science Distributed Systems E-commerce Engineering Git +5
401(k) matching Career development Equity Flexible spending account Flex vacation +6
Columbus, OH, United States Columbus, OH, United States Full TimeSenior Senior-levelUSD 115K - 180K * USD 115K+ *
Lead Data Engineer (AWS, Azure, GCP)AWS Azure Big Data BigQuery CI/CD Databricks Data pipelines +25
Career development Fertility benefits Health care Insurance Wellness
Newark, NJ, United States Newark, NJ, United States Full TimeSenior Senior-levelUSD 115K - 180K * USD 115K+ *
Lead ETL Data EngineerBig Data Computer Science Data analysis Data governance Data management Data pipelines Data quality +16
Career development Equity Insurance Startup environment Team events
Jersey City, NJ, United … Jersey City, NJ, United States Full TimeSenior Senior-levelUSD 69K - 128K * USD 69K+ *
Lead Data Visualization EngineerBig Data Computer Science Data analysis Data governance Data management Data pipelines Data quality +16
Career development Equity Insurance Startup environment Team events
Explore more AI/ML/Data Science career opportunities
Find open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general, filtered by job title or popular skill, toolset and products used.
- Open BI Developer jobs
- Open Junior Data Analyst jobs
- Open Data Science Intern jobs
- Open Staff Data Scientist jobs
- Open Director, Data Engineering jobs
- Open Junior Data Engineer jobs
- Open Product Data Analyst jobs
- Open Senior Data Analyst (Bangkok Based, relocation provided) jobs
- Open Power BI Developer jobs
- Open Junior Data Scientist jobs
- Open Associate Data Analyst- Customer Experience Group | Bangkok-based jobs
- Open Data Analyst (Remote) jobs
- Open Marketing Data Analyst jobs
- Open Lead Data Analyst jobs
- Open Data Analytics Manager jobs
- Open Head of Data Science jobs
- Open Machine Learning Scientist jobs
- Open Data Analyst (Statistics/Python/BI) (Bangkok-based, relocation provided) jobs
- Open Big Data Engineer jobs
- Open Data Manager jobs
- Open Sr. Data Analyst jobs
- Open Computer Vision Engineer jobs
- Open Data Scientist (Remote) jobs
- Open Data Engineer Intern jobs
- Open Autonomous Vehicle System Test Specialist jobs
- Open APIs-related jobs
- Open Excel-related jobs
- Open ML models-related jobs
- Open Data quality-related jobs
- Open Finance-related jobs
- Open Airflow-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Scala-related jobs
- Open Business Intelligence-related jobs
- Open Hadoop-related jobs
- Open Data visualization-related jobs
- Open Kafka-related jobs
- Open Data warehouse-related jobs
- Open Git-related jobs
- Open Docker-related jobs
- Open Kubernetes-related jobs
- Open DevOps-related jobs
- Open TensorFlow-related jobs
- Open PyTorch-related jobs
- Open Streaming-related jobs
- Open NLP-related jobs
- Open NoSQL-related jobs