Sr. Data Engineer
Dallas, TX
Company
Federal Reserve Bank of DallasWe are dedicated to serving the public by promoting a strong financial system and a healthy economy for all. These efforts take a team of dedicated individuals doing many different jobs. Together we’re creating a workplace where talented people can thrive, and we welcome your unique background and perspective to help present the best possible solutions for our partners.The Data Engineer responsible for designing, developing, and maintaining data pipelines in support of data engineering and data management activities. You must be passionate about data engineering and data quality with a solid background in AWS, Databricks, Python, Trino/Starburst and SQL.
Location: #LI-Hybrid
About the Role:
The Sr. Data Engineer develops data solutions with moderate to high complexity that also have moderate to high business criticality. The Engineer also develops data set processes; works towards improving data, efficiency, and quality; uses large datasets to address business issues; provides full software development lifecycle support; manages design, security, standards, data quality and compliance processes; works independently with guidance only in the most complex situations. The engineer may lead functional teams or projects, has in-depth domain knowledge of the relevant industry. Other responsibilities include requirement analysis, design, code, test, debug, document and maintain data and analytics solutions.
You will report to the Technology Solutions Manager in Dallas who manages an Engineering team consisting of 10 team members with assignments in Dallas and New Yor City. We are a collaborative, passionate team delivering sustainable software and data-driven solutions that meet the needs of our customers across the Federal Reserve System.
You Will:
- Design, develop, monitor, and maintain data pipelines in an AWS Gov Cloud ecosystem with Databricks, Delta Lake and Starburst as the underlying platforms
- Collaborate with cross-functional teams to understand data needs and translate them into effective data pipeline solutions.
- Develop, optimize, and maintain ETL processes to facilitate the smooth and accurate movement of data across systems.
- Establish data quality checks and ensure data integrity and accuracy throughout the data lifecycle.
- Implement and enforce data governance policies and procedures.
- Optimize data processing and query performance for large-scale datasets within AWS and Databricks environments.
- Document data engineering processes, architecture, and configurations.
- Troubleshooting and debugging data-related issues on the AWS Databricks platform.
- Having the knowledge about Optimizing AWS Databricks Streaming jobs.
- Integrating Databricks with other data storage and processing systems.
You Have:
- Bachelor’s or master’s degree in computer science, Information Technology, or a related field.
- Minimum of 5 years of experience in data engineering roles, with a focus on AWS and Databricks.
- Highly proficient with Apache Spark, Starburst/Trino, Python, PySpark and SQL
- Hands-on experience in Gitlab with CI/CD.
- Hands-on experience in AWS Services like S3, RDS, Lambda, SQS, SNS, MSK is required.
- Have a deep understanding of AWS Cloud platforms and data engineering technologies.
- Strong SQL skills to perform data analysis and understanding of source data.
- Familiarity with NoSQL databases and JSON formats.
- Good to have experience with ETL tools.
- Experience with data modeling, schema design, and database optimization.
- Experience with data pipeline orchestration tools.
- Ability to troubleshoot complex data issues and implement effective solutions.
- Strong communication and interpersonal skills.
- Ability to work collaboratively in a team-oriented environment.
- Proactive in staying updated with industry trends and emerging technologies in data engineering.
Our Benefits:
Our total rewards program offers benefits that are the best fit for you at every stage of your career:
- Comprehensive healthcare options (Medical, Dental, and Vision)
- 401K match, and a fully funded pension plan
- Paid vacation, holidays, and volunteer hours; flexible work environment
- Generously subsidized public transportation and free parking
- Annual tuition reimbursement
- Professional development programs, training, and conferences
- And more…
Notes:
This position may be filled at various levels based on candidate's qualifications as determined by the department.
This position requires access to confidential supervisory information and/or FOMC information, which is limited to "Protected Individuals" as defined in the U.S. federal immigration law. Protected Individuals include, but are not limited to, U.S. citizens, U.S. nationals, and U.S. permanent residents who either are not yet eligible to apply for naturalization or who have applied for naturalization within the requisite timeframe. Candidates who are permanent residents may be eligible for the information access required for this position if they sign a declaration of intent to become a U.S. citizen and pursue a path to citizenship and meet other eligibility requirements. In addition, all candidates must undergo an enhanced background check and comply with all applicable information handling rules.
This role allows for the employee to work in a hybrid manner with some work conducted onsite at the Federal Reserve work location and some work conducted remotely. Specific schedule details will be discussed during the recruitment process.
The Federal Reserve Bank of Dallas is proud to be an Equal Opportunity Employer that believes in the diversity of our people, ideas and experiences, and we are committed to building an inclusive culture that represents the communities we serve.
If you need assistance or an accommodation due to a disability, please notify your Talent Acquisition Consultant.
Full Time / Part Time
Full timeRegular / Temporary
RegularJob Exempt (Yes / No)
YesJob Category
Work Shift
First (United States of America)The Federal Reserve Banks believe that diversity and inclusion among our employees is critical to our success as an organization, and we seek to recruit, develop and retain the most talented people from a diverse candidate pool. The Federal Reserve Banks are committed to equal employment opportunity for employees and job applicants in compliance with applicable law and to an environment where employees are valued for their differences.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Architecture AWS CI/CD Computer Science Data analysis Databricks Data governance Data management Data pipelines Data quality Engineering ETL GitLab JSON Lambda NoSQL Pipelines Privacy PySpark Python Security Spark SQL Streaming
Perks/benefits: 401(k) matching Career development Conferences Flex hours Flex vacation Health care Team events
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Marketing Data Analyst jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Principal Data Engineer jobs
- Open Data Manager jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Sr Data Engineer jobs
- Open Business Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Data Analyst Intern jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Principal Data Scientist jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open Research Scientist jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Business Intelligence-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs