Senior Data Engineer with Databricks - Empower (remote/US-based)
Denver, CO, United States
Applications have closed
Hitachi Solutions
Company Description
Company Overview
Hitachi Solutions is a global solutions integrator passionate about designing, developing, and delivering cutting edge cloud solutions to help our clients innovative across their entire business. Our firm develops the business services and technology powering some of the products you use every day – and is closely aligned with Microsoft and other leaders in the cloud computing space.
What sets Hitachi Solutions apart is both our industry focus, and the intellectual property that we bring to our customers. Recognized for our achievements year after year, we strive to be the trusted advisor of large and medium sized enterprises alike – helping them move fast to achieve strategic business initiatives with distinguished engineering, hard work, and compassion. With over 3,000 team members across 14 countries, in our 18 years of focus our company has seen explosive growth and high customer satisfaction. This has allowed us to offer exceptionally compelling salaries, 401k match, family leave, and health benefits. And no – we will not make you come into an office or ask for an inflexible work schedule.
A part of Hitachi, Ltd., our company has a long and rich history of innovation, financial strength, and international presence of one of the world’s largest companies. Since 1910, Hitachi, Ltd. has been a leader in manufacturing innovative products and solutions that support industry and social infrastructure around the globe supported by 303,000 employees in over 100 countries and across 864 companies.
New Product Development and Innovations Team
This position in our company is housed in our New Product Development and Innovations team formed in 2021. Joining this team represents an opportunity to fast-track your career and to work with a team of fun and nerdy colleagues in a disruptive atmosphere: well-funded, focused on hypergrowth, moving quickly, and making mistakes in the furtherance of innovation and sound engineering.
Armed with an existing book of business, and a stable financial parent – it is the goal of this group to help our firm introduce products to enhance our already strong services business – ultimately making the cloud easier for our customers, and allowing us to hit our long term financial goals with greater-than linear scale.
Job Description
Please note: Although our position is remote / virtual / work-from-home, you MUST reside, and be authorized to work, in the US or Canada.
SENIOR DATA ENGINEER (DATABRICKS, PYTHON, SPARK)
This is a full-time role in our product organization for a highly experienced Data Engineer in Big Data systems design with skills in data architecture, especially Spark and Delta/Data Lake technology.
Individuals in this role will assist in the design, development, enhancement, and maintenance of complex data pipelines products that manage business critical operations, and large-scale analytics pipelines. Qualified applicants will have a demonstrated capability to learn new concepts quickly, have a data engineering background, and/or have robust software engineering expertise.
Responsibilities
- Scope and execute together with team leadership. Work with the team to understand platform capabilities and how to best improve and expand those capabilities.
- Strong independence and autonomy.
- Design, development, enhancement, and maintenance of complex data pipeline products which manage business-critical operations and large-scale analytics applications.
- Experience leading mid- and senior-level data engineers.
- Support analytics, data science and/or engineering teams and understand their unique needs and challenges.
- Instill excellence into the processes, methodologies, standards, and technology choices embraced by the team.
- Embrace new concepts quickly to keep up with fast-moving data engineering technology.
- Dedicate time to continuous learning to keep the team appraised of the latest developments in the space.
- Commitment to developing technical maturity across the company.
Qualifications
- 5+ years of Data Engineering experience including 2+ years designing and building Databricks data pipelines is REQUIRED; Azure cloud preferred, will consider AWS, GCP or other cloud platform experience in lieu of
- Experience with conceptual, logical and/or physical database designs is HIGHLY DESIRED
- 2+ years of hands-on Python/Pyspark/SparkSQL experience is REQUIRED; experience with Scala is a plus
- 2+ years of experience with Big Data pipelines or DAG Tools (Airflow, Dbt, Data Factory, or similar) is REQUIRED
- 2+ years of Spark experience (especially Databricks Spark and Delta Lake) is REQUIRED
- 2+ years of hands-on experience implementing Big Data solutions in a cloud ecosystem, including Data/Delta Lakes, is REQUIRED
- 2+ years of experience with source control (git) on the command line is REQUIRED
- 2+ years of SQL experience, specifically to write complex, highly optimized queries across large volumes of data is HIGHLY DESIRED
- Strong data modeling / data profiling capabilities with Kimball/star schema methodology is HIGHLY DESIRED
- Professional experience with Kafka or other live streaming technology is HIGHLY DESIRED
- Professional experience with database deployment pipelines (i.e., dacpac’s or similar technology) is HIGHLY DESIRED
- Professional experience with one or more unit testing or data quality frameworks is HIGHLY DESIRED
#LI-CA1
#REMOTE
#DATABRICKS
#SPARK
#DATALAKEHOUSE
Additional Information
We are an equal opportunity employer. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow Architecture AWS Azure Big Data Databricks Data pipelines Data quality Engineering GCP Git Kafka Pipelines PySpark Python Scala Spark SQL Streaming Testing
Perks/benefits: 401(k) matching Career development Health care Startup environment
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open AI Engineer jobs
- Open Data Engineer II jobs
- Open Sr Data Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Principal Data Engineer jobs
- Open Data Manager jobs
- Open Data Analytics Engineer jobs
- Open Power BI Developer jobs
- Open Junior Data Scientist jobs
- Open Senior Data Architect jobs
- Open Product Data Analyst jobs
- Open Data Scientist II jobs
- Open Business Intelligence Developer jobs
- Open Sr. Data Scientist jobs
- Open Manager, Data Engineering jobs
- Open Big Data Engineer jobs
- Open Business Data Analyst jobs
- Open Data Quality Analyst jobs
- Open Principal Data Scientist jobs
- Open ETL Developer jobs
- Open Data Analyst Intern jobs
- Open Junior Data Engineer jobs
- Open Research Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Java-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open TensorFlow-related jobs
- Open Snowflake-related jobs
- Open PhD-related jobs
- Open NLP-related jobs
- Open CI/CD-related jobs
- Open Data governance-related jobs
- Open Kubernetes-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs
- Open Hadoop-related jobs
- Open LLMs-related jobs
- Open Data warehouse-related jobs