Data Engineer - Google Cloud
USA - Remote
Quantiphi
While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Experience Level: 4+ years
Responsibilities:
- Work with cloud engineers and customers to solve big data problems by developing utilities for migration, storage and processing on Cloud platforms.
- Design and build a cloud migration strategy for cloud and on-premise applications.
- Diagnose and troubleshoot complex distributed systems problems and develop solutions with a significant impact at massive scale.
- Build tools to ingest and jobs to process several terabytes or petabytes per day.
- Design and develop next-gen storage and compute solutions for several large customers.
- Communicate with a wide set of teams, including Infrastructure, Network, Engineering, DevOps, SiteOps teams, and cloud customers.
- Build advanced tooling for automation, testing, monitoring, administration, and data operations across multiple cloud clusters.
Required Skills:
- Hands-on experience in data structures, distributed systems, Hadoop and spark,
- SQL and NoSQL Databases
- Strong software development skills in at least one of: Python, PySpark.
- Experience in developing Big Data solutions (migration, storage, processing)
- Experience building and supporting large-scale systems in a production environment
- Cloud Platforms – AWS, GCP or Azure Big Data Distributions
- Any of Apache Hadoop/CDH/HDP/EMR/Google DataProc/HD-Insights Distributed processing Frameworks.
- One or more of MapReduce, Apache Spark, Apache Storm, Apache Flink.
- Database/warehouse
- Hive, HBase, and at least one cloud native services Orchestration Frameworks
- Any of Airflow, Oozie, Apache NiFi, Google DataFlow Message/Event Solutions
- Any of Kafka, Kinesis, Cloud pub-sub Container Orchestration (Good to have)
- Kubernetes or Swarm
- Leadership qualities
- Ability to lead technology teams and provide them mentorship / support to accelerate performance.
- Experience in leading multiple large projects as well as a deep understanding of Agile developments
- Effective communication with all the stakeholders involved.
- Communicate clearly about complex subjects and technical plans with technical and non technical audiences.
What is in it for you:
- Opportunity to learn cloud-native services and how to utilize those services to solve various business problems.
- More hands-on learning opportunities in Python, SQL, etc
- Sponsored certification opportunity for various courses of your choice (eg, GCP, AWS, Azure, Tableau, Looker, etc)
- Allocated work will allow you to not only just complete the task but you will be given an opportunity to own up the work and take responsibility for it’s end-to-end delivery
If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow AWS Azure Big Data Dataflow DataOps Dataproc DevOps Distributed Systems Engineering Excel Flink GCP Google Cloud Hadoop HBase Kafka Kinesis Kubernetes Looker NiFi NoSQL Oozie PySpark Python Spark SQL Tableau Testing
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Lead Data Analyst jobs
- Open MLOps Engineer jobs
- Open Data Manager jobs
- Open Senior Business Intelligence Analyst jobs
- Open Principal Data Engineer jobs
- Open Data Engineer II jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Data Scientist II jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Data Analyst Intern jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Junior Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Data Quality Analyst jobs
- Open Azure Data Engineer jobs
- Open Data Product Manager jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Business Intelligence-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open Deep Learning-related jobs
- Open PhD-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open CI/CD-related jobs
- Open Generative AI-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open DevOps-related jobs