Senior Data Engineer - Cybersecurity
Tempe, AZ, United States
Applications have closed
Silicon Valley Bank
Silicon Valley Bank is the financial partner of the innovation economy – helping individuals, investors and the world’s most innovative companies achieve extraordinary outcomes.Company Description
For over 30 years, Silicon Valley Bank (SVB) has helped innovative companies and their investors move bold ideas forward, fast. SVB provides targeted banking services to companies of all sizes in innovation centers around the world.
Job Description
When you work with the world's most innovative companies, you know you're making a difference.
Our clients are the game changers, leaders and investors who fuel the global innovation economy. They're the businesses behind the next medical breakthroughs. And the visionaries whose new technologies could transform the way people live and work.
They come to SVB for our expertise, deep network and nearly forty years of experience in the industries we serve, and to partner with diverse teams of passionate, enterprising SVBers, dedicated to an inclusive approach to helping them grow and succeed at every stage of their business.
Join us at SVB and be part of bringing our clients' world-changing ideas to life. At SVB, we have the opportunity to grow and collectively make an impact by supporting the innovative clients and communities SVB serves. We pride ourselves in having both a diverse client roster and an equally diverse and inclusive organization. And we work diligently to encourage all with different ways of thinking, different ways of working, and especially those traditionally underrepresented in technology and financial services, to apply.
Qualifications
Essential skills needed include Scripting in Python, SQL, Shell, Yaml (for CICD), AWS Glue/PySpark, AWS Athena, AWS Lambda/Python, AWS Lakeformation
Advantageous skills would include Databricks, EMR, & Big Data technologies
Advanced understanding of both SQL and NoSQL technologies
Understand and implement secure coding practices
6+ years as a Python, PySpark, SQL developer; building scalable ETL applications and data warehouses
Advanced proficiency programming in Python & PySpark ETL modules is required
Experience in working with and processing large data sets in a time-sensitive environment while minimizing errors
Hands-on experience working on On-premise and Cloud data processing/movement solutions.
Hands-on experience working with big data technologies (Hadoop, Hive, Spark)
Proficient experience working within the AWS and AWS tools (S3, Glue, EMR, Athena, etc)
Experienced in maintaining infrastructure as code using Terraform or cloud formation
Experienced in building Data Visualizations using automations. Hands-on experience working with Tableau and BI tools
Solid understanding of data warehouse design patterns and best practices
Ability to develop test plans and stress test platforms
Experience with complex Job scheduling
Effective analytical, conceptual, and problem-solving skills
Must be organized, disciplined, and task/goal oriented
Able to prioritize and coordinate work through interpretation of high-level goals and strategy
Effective team player with a positive attitude
Strong oral and written English language communications skills
Responsibilities
The Data Engineer is responsible for operationalizing data pipelines that support metrics & analytics initiatives for the company.
The primary responsibilities include designing, building, managing, optimizing and documenting data flows from various sources into our enterprise data lake
Delivery of high-quality data is a key item of focus
The data engineer is expected to collaborate with data scientists, data analysts and other data consumers to productionize data models and algorithms developed by those users to improve the overall efficiency of advanced analysis projects
Additionally, the data engineer is responsible for ensuring data quality, governance and data security procedures are met while curating data for use in the Data Lake
Design and develop Lambda and AWS Batch scripts in Python
Design and incorporate error handling & Data Quality processes into pipelines and processes
Design, implement, and analyze robust test plans and stress tests
End-to-end Implementation and monitoring of Data Pipelines
Lead and/or work with cross-disciplinary teams to understand, document and analyze customer needs
Identify and present a range of potential solution options for any demand, informing stakeholders of advantages and disadvantages of each; assist them in arriving at an optimal solution strategy
Optimize flexibility, scalability, performance, reliability, and future-proof capacity of IT services, at an optimal cost
Implement chosen solutions, including infrastructure, scripts, database resources, permissions, source control
Contribute to the wider enterprise architecture and roadmap
Conduct research into, test, and trial new technologies and approaches they could enhance our work
Educate and train yourself and others as you evangelize the merits of data and analytics
Document own, or existing projects, in a clear yet comprehensive format for a wide range of audiences
Contribute to enhancing the team's own internal processes of communications, documentation, workload planning
Work closely with management to prioritize business and information request backlogs
Additional Information
Locations: Austin, TX/ Tempe, AZ/ Atlanta, GA/ Charlotte, NC.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Architecture Athena AWS AWS Glue Banking Big Data Databricks Data pipelines Data quality Data warehouse ETL Hadoop Lambda NoSQL Pipelines PySpark Python Research Security Spark SQL Tableau Terraform
Perks/benefits: Health care
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open MLOps Engineer jobs
- Open Lead Data Analyst jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Data Manager jobs
- Open Sr Data Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Power BI Developer jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Data Scientist II jobs
- Open Senior Data Architect jobs
- Open Product Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Business Data Analyst jobs
- Open Manager, Data Engineering jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Data Quality Analyst jobs
- Open Data Product Manager jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open ETL Developer jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Java-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Kubernetes-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs
- Open LLMs-related jobs
- Open DevOps-related jobs