Senior Data Engineer
San Francisco, United States
Applications have closed
Atlassian
Atlassian's team collaboration software like Jira, Confluence and Trello help teams organize, discuss, and complete shared work.Atlassian can hire people in any country where we have a legal entity. Assuming you have eligible working rights and a sufficient time zone overlap with your team, you can choose to work remotely or return to an office as they reopen (unless it’s necessary for your role to be performed in the office). Interviews and onboarding are conducted virtually, a part of being a distributed-first company.
JOB DUTIES: DESIGN EVENT COLLECTION INFRASTRUCTURE AND BUILD DATA MODELS APPLYING A DEEP UNDERSTANDING OF BIG DATA CHALLENGES AND ECO-SYSTEM. APPLY KNOWLEDGE OF EXTRACT, TRANSFORM AND LOAD (ETL) PROCESSES TO COLLECT, EXTRACT, AND CLEAN THE DATA FOR SUBSEQUENT REPORTING AND ANALYSIS. PARTNER WITH ATLASSIAN TECHNOLOGY TEAMS, ANALYTICAL TEAMS, AND DATA SCIENTIST ACROSS VARIOUS INITIATIVES TO MAKE DATA MODEL MORE SCALABLE, RELIABLE, MAINTAINABLE, AND BETTER INTEGRATED WITH OTHER PARTS OF COMPANY’S DATA ECOSYSTEM. DESIGN, DEVELOP AND BUILD WORLD-CLASS DATA SOLUTIONS AND SOFTWARE APPLICATIONS. WORK WITH LARGE DATASETS, ARCHITECT/DESIGN, BUILD, LAUNCH, AS WELL AS MANAGE DATA MODELS AND PROCESSES TO PROVIDE SOFTWARE DATA FOR THE ANALYTICS TEAM AND FOR BUSINESS USE. DESIGN, BUILD, AND MANAGE DATA WAREHOUSE AS WELL AS CREATE SOFTWARE PLATFORMS REPORT AND DATA VISUALIZATION. RESPONSIBLE FOR IMPROVING THE DATASETS BY ADDING NEW SOURCES, CODING BUSINESS RULES, AND PRODUCING NEW METRICS THAT SUPPORT THE BUSINESS. DEVELOP INFORMATION ARCHITECTURE, DATA MODELS AND EXTRACT TRANSFORM LOAD (ETL) PROCESSES TO CONSUME MARKETING DATA SOURCED FROM MULTIPLE BIG DATA DEPOSITORIES, INCLUDING SPARK/HADOOP AND REDSHIFT, BUILD AND DESIGN SOFTWARE DATA AND APPLICATION SOLUTIONS WITH PUBLIC CLOUD OFFERINGS SUCH AS AMAZON WEB SERVICES, REDSHIFT, S3, EMR/SPARK, PRESTO/ATHENA, AND HIVE. DESIGN, BUILD, IMPROVE DATA INFRASTRUCTURES, AND MANAGE DATA PIPELINES AND ETL PROCESSES USING PROGRAMMING LANGUAGES SUCH AS SQL, SQL TUNING, SCHEMA DESIGN, AND PYTHON. MANAGE COMPLEX WORKFLOWS USING OPEN SOURCE DATA PIPELINE SUCH AS WORKFLOW TOOLS AS AIRFLOW, OOZIE OR LUIGI. AUTOMATE DATA PIPELINE AND REPORTING PROCESSES AND ENSURE DATA QUALITY ACROSS MULTIPLE DATASETS. MINIMUM REQUIREMENTS: BACHELOR’S DEGREE IN COMPUTER SCIENCE OR A RELATED FIELD OF STUDY PLUS FIVE (5) YEARS’ PROGRESSIVE DATA ENGINEERING EXPERIENCE INCLUDING EXPERIENCE WITH SOLUTION BUILDING AND ARCHITECTING WITH PUBLIC CLOUD OFFERINGS SUCH AS AMAZON WEB SERVICES OR SIMILAR DISTRIBUTED SYSTEMS S3 OR HDFS, EMR/SPARK OR HADOOP, PRESTO/ATHENA OR SIMILAR DATABASE TECHNOLOGIES; SPARK OR HIVE; SQL, PYTHON OR JAVA AND ETL PROCESSES; DATA PIPELINE WITH WORKFLOW TOOLS SUCH AS AIRFLOW, OOZIE OR LUIGI; BUILDING RESTFUL APIS AND MICROSERVICES, E.G. WITH FLASK; TEST AUTOMATION AND ENSURING DATA QUALITY ACROSS MULTIPLE DATASETS USED FOR ANALYTICAL PURPOSE; LAMBDA ARCHITECTURE OR OTHER BIG DATA ARCHITECTURAL BEST PRACTICES; TEST AUTOMATION AND CONTINUOUS DELIVERY. ALTERNATE REQUIREMENTS: MASTER’S DEGREE IN COMPUTER SCIENCE OR A RELATED FIELD OF STUDY PLUS THREE (3) YEARS’ DATA ENGINEERING EXPERIENCE INCLUDING EXPERIENCE WITH SOLUTION BUILDING AND ARCHITECTING WITH PUBLIC CLOUD OFFERINGS SUCH AS AMAZON WEB SERVICES OR SIMILAR DISTRIBUTED SYSTEMS S3 OR HDFS, EMR/SPARK OR HADOOP, PRESTO/ATHENA OR SIMILAR DATABASE TECHNOLOGIES; SPARK OR HIVE; SQL, PYTHON OR JAVA AND ETL PROCESSES; DATA PIPELINE WITH WORKFLOW TOOLS SUCH AS AIRFLOW, OOZIE OR LUIGI; BUILDING RESTFUL APIS AND MICROSERVICES, E.G. WITH FLASK; TEST AUTOMATION AND ENSURING DATA QUALITY ACROSS MULTIPLE DATASETS USED FOR ANALYTICAL PURPOSE; LAMBDA ARCHITECTURE OR OTHER BIG DATA ARCHITECTURAL BEST PRACTICES; TEST AUTOMATION AND CONTINUOUS DELIVERY. SPECIAL REQUIREMENTS: MUST PASS TECHNICAL INTERVIEW. Our perks & benefits
To support you at work and play, our perks and benefits include ample time off, an annual education budget, paid volunteer days, and so much more.
About Atlassian
The world’s best teams work better together with Atlassian. From medicine and space travel, to disaster response and pizza deliveries, Atlassian software products help teams all over the planet. At Atlassian, we're motivated by a common goal: to unleash the potential of every team.
We believe that the unique contributions of all Atlassians create our success. To ensure that our products and culture continue to incorporate everyone's perspectives and experience, we never discriminate based on race, religion, national origin, gender identity or expression, sexual orientation, age, or marital, veteran, or disability status. All your information will be kept confidential according to EEO guidelines.
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
To learn more about our culture and hiring process, explore our Candidate Resource Hub.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow APIs Athena Big Data Computer Science Data pipelines Data visualization Distributed Systems Engineering ETL Flask Hadoop HDFS Lambda Microservices Oozie Open Source Pipelines Python Redshift Spark SQL
Perks/benefits: Travel
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open AI Engineer jobs
- Open Data Science Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Principal Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Scientist II jobs
- Open Junior Data Scientist jobs
- Open Product Data Analyst jobs
- Open Senior Data Architect jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Big Data Engineer jobs
- Open Data Analyst Intern jobs
- Open Manager, Data Engineering jobs
- Open Azure Data Engineer jobs
- Open Junior Data Engineer jobs
- Open Data Product Manager jobs
- Open Data Quality Analyst jobs
- Open Principal Data Scientist jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open GCP-related jobs
- Open Data management-related jobs
- Open Java-related jobs
- Open Privacy-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open APIs-related jobs
- Open Deep Learning-related jobs
- Open PyTorch-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open TensorFlow-related jobs
- Open PhD-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open NLP-related jobs
- Open Data governance-related jobs
- Open LLMs-related jobs
- Open Airflow-related jobs
- Open Data warehouse-related jobs
- Open Hadoop-related jobs
- Open Databricks-related jobs