Senior Data Engineer
Remote
Responsibilities:
- Creates new pipeline development and maintains existing pipeline, updates Extract, Transfer, Load (ETL) process, creates new ETL feature development
- Supports software developers, database architects, data analysts and data scientists on data initiatives and ensure optimal data delivery architecture is consistent throughout ongoing projects.
- Assembles large, complex sets of data that meet non-functional and functional business requirements
- Builds required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies
- Builds analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
- Works with stakeholders including data, design, product and government stakeholders and assisting them with data-related technical issues
- Writes unit and integration tests for all data processing code.
- Works with DevOps engineers on CI, CD, and IaC.
- Reads specs and translate them into code and design documents.
Required Qualifications:
- Minimum of 8 years related experience.
- 4 years of hands-on software development experience
- 4 years of Data pipeline experience using Python, Java and cloud technologies
- A Bachelor’s degree in Computer Science, Information Systems, Engineering, Business, or other related scientific or technical discipline. With ten years of general information technology experience and at least eight years of specialized experience, a degree is NOT required.
- Experienced in data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.
- Experienced in designing data services including API, meta data, and data catalog.
- Experienced in data governance process to ingest (batch, stream), curate, and share data with upstream and downstream data users.
- Familiar with work to build and optimize data sets, ‘big data’ data pipelines and architectures
- Familiar with work to perform root cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions
- Analytic skills associated with working on unstructured datasets
- Familiar with work to build processes that support data transformation, workload management, data structures, dependency and metadata
- Demonstrated understanding using software and tools including big data tools like Kafka, Spark and Hadoop; relational NoSQL and SQL databases including Cassndra and Postgres; workflow management and pipeline tools such as Airflow, Luigi and Azkaban; AWS cloud services including Redshift, RDS, EMR and EC2; stream-processing systems like Spark-Streaming and Storm; and object function/object-oriented scripting languages including Scala, C++, Java and Python.
- Flexible and willing to accept a change in priorities as necessary.
- Ability to work in a fast-paced, team-oriented environment
- Experience with Agile methodology, using test-driven development.
- Experience with Atlassian Jira/Confluence.
- Excellent command of written and spoken English.
- Ability to obtain and maintain a Public Trust; residing in the United States
Desired Qualifications:
- Federal Government contracting work experience.
- Google’s Certified Professional-Data-Engineer certification, IBM Certified Data Engineer – Big Data certification, CCP Data Engineer for Cloudera
- Centers for Medicare and Medicaid Services (CMS) or Health Care Industry experience
- Experience with healthcare quality data including Medicaid and CHIP provider data, beneficiary data, claims data, and quality measure data.
- Experienced in designing data architecture for shared services, scalability, and performance
Occasional travel for training and project meetings. It is estimated to be 5-15% per year.
Benefits:We offer highly competitive salary, full healthcare benefits and a flexible leave policy.
Equal Employment Opportunity:eSimplicity is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, gender, age, status as a protected veteran, sexual orientation, gender identity, or status as a qualified individual with a disability.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow APIs Architecture AWS Azkaban Big Data Computer Science Confluence Data governance Data pipelines DevOps EC2 Engineering ETL Hadoop Java Jira Kafka NoSQL Pipelines PostgreSQL Python Redshift Scala Spark SQL Streaming TDD
Perks/benefits: Career development Competitive pay Flex hours Health care Startup environment
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Lead Data Analyst jobs
- Open Data Science Manager jobs
- Open Marketing Data Analyst jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open MLOps Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Scientist II jobs
- Open Power BI Developer jobs
- Open Business Intelligence Developer jobs
- Open Data Analytics Engineer jobs
- Open Business Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Data Analyst Intern jobs
- Open Sr Data Engineer jobs
- Open Product Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Research Scientist jobs
- Open Big Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Principal Data Scientist jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Data Product Manager jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open PyTorch-related jobs
- Open NLP-related jobs
- Open TensorFlow-related jobs
- Open APIs-related jobs
- Open Consulting-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Snowflake-related jobs
- Open Generative AI-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Docker-related jobs
- Open Airflow-related jobs