Principal Developer - Data Integration Engineer
San Jose, CA, United States
Applications have closed
Western Digital
Western Digital, leaders in digital storage solutions compatible with Mac and PC. FREE shipping, friendly support, and 30-day return policy on storage products.Company Description
At Western Digital, our vision is to power global innovation and push the boundaries of technology to make what you thought was once impossible, possible.
At our core, Western Digital is a company of problem solvers. People achieve extraordinary things given the right technology. For decades, we’ve been doing just that. Our technology helped people put a man on the moon.
We are a key partner to some of the largest and highest growth organizations in the world. From energizing the most competitive gaming platforms, to enabling systems to make cities safer and cars smarter and more connected, to powering the data centers behind many of the world’s biggest companies and public cloud, Western Digital is fueling a brighter, smarter future.
Binge-watch any shows, use social media or shop online lately? You’ll find Western Digital supporting the storage infrastructure behind many of these platforms. And, that flash memory card that captures and preserves your most precious moments? That’s us, too.
We offer an expansive portfolio of technologies, storage devices and platforms for business and consumers alike. Our data-centric solutions are comprised of the Western Digital®, G-Technology™, SanDisk® and WD® brands.
Today’s exceptional challenges require your unique skills. It’s You & Western Digital. Together, we’re the next BIG thing in data.
Job Description
ESSENTIAL DUTIES AND RESPONSIBILITIES:
- Develop, test, and deploy data integration pipeline jobs using chosen cloud based native tool according to Western Digital design principles.
- Troubleshoot existing pipeline jobs based on BASH, Python, Java and modify jobs for performance improvements.
- Develop new pipelines using Java/Spark/Scala on AWS EMR platform and schedule using AirFlow
- Design data tables for faster load and optimal query performance
- Debug SQL statements and stored procedures in Postgres and Redshift databases.
- Use git-based source control system bit bucket and continuous integration tools like Jenkins and deployment using Ansible.
Qualifications
REQUIRED EDUCATIONAL BACKGROUND
- Minimum of a bachelor’s degree in computer science or engineering. Master’s degree preferred.
- AWS developer certification will be preferred.
- Any certification on SDLC (Software Development Life Cycle) methodology, integrated source control system, continuous development and continuous integration will be preferred
REQUIRED PROFESSIONAL BACKGROUND
- Minimum of 8+ years of working in Data Integration/Engineering using any cloud/on-premises ETL tools
- Passionate about working with BASH, Python and Java. Experience in running Spark/Scala programs, Java UDFs in AWS EMR platform is preferred.
- Understanding on Unix/Linux operating system like awk, ssh, crontab, etc.,
- Ability to write transact SQL, develop and debug stored procedures and user defined functions in python.
- Working experience on Postgres and/or Redshift database is preferred. Concepts of MPP, Massively Parallel Processing well understood and practiced in solving business problems.
- Exposure to CI/CD tools like bit bucket, Jenkins, ansible, docker, etc. is preferred.
- Ability to understand relational database systems and its concepts.
- Understanding of Semiconductor manufacturing related datasets and concepts preferred
REQUIRED PERSONAL BACKGROUND
- Passionate about working with large datasets
- Ability to read, write and speak in English clearly
- Ability to work as a team
- Ability to work in multiple projects and cope up with tight deadline and time constraints
- Ability to freely express his own view and accept well known methodologies.
Additional Information
Western Digital is committed to providing equal opportunities to all applicants and employees and will not discriminate based on their race, color, ancestry, religion (including religious dress and grooming standards), sex (including pregnancy, childbirth or related medical conditions, breastfeeding or related medical conditions), gender (including a person’s gender identity, gender expression, and gender-related appearance and behavior, whether or not stereotypically associated with the person’s assigned sex at birth), age, national origin, sexual orientation, medical condition, marital status (including domestic partnership status), physical disability, mental disability, medical condition, genetic information, protected medical and family care leave, Civil Air Patrol status, military and veteran status, or other legally protected characteristics. We also prohibit harassment of any individual on any of the characteristics listed above. Our non-discrimination policy applies to all aspects of employment. We comply with the laws and regulations set forth in the Equal Employment Opportunity is the Law poster.
Western Digital thrives on the power and potential of diversity. As a global company, we believe the most effective way to embrace the diversity of our customers and communities is to mirror it from within. We believe the fusion of various perspectives results in the best outcomes for our employees, our company, our customers, and the world around us. We are committed to an inclusive environment where every individual can thrive through a sense of belonging, respect and contribution.
Western Digital is committed to offering opportunities to applicants with disabilities and ensuring all candidates can successfully navigate our careers website and our hiring process. Please contact us at jobs.accommodations@wdc.com to advise us of your accommodation request. In your email, please include a description of the specific accommodation you are requesting as well as the job title and requisition number of the position for which you are applying.
#LI-AS1
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow Ansible AWS CI/CD Computer Science Docker Engineering ETL Git Java Linux MPP Pipelines PostgreSQL Python RDBMS Redshift Scala SDLC Spark SQL
Perks/benefits: Career development Medical leave
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open MLOps Engineer jobs
- Open Lead Data Analyst jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Manager jobs
- Open Data Engineer II jobs
- Open Principal Data Engineer jobs
- Open Power BI Developer jobs
- Open Sr Data Engineer jobs
- Open Business Intelligence Developer jobs
- Open Junior Data Scientist jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Data Scientist II jobs
- Open Business Data Analyst jobs
- Open Senior Data Architect jobs
- Open Sr. Data Scientist jobs
- Open Data Analyst Intern jobs
- Open Big Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Data Quality Analyst jobs
- Open Data Product Manager jobs
- Open Junior Data Engineer jobs
- Open Principal Data Scientist jobs
- Open Azure Data Engineer jobs
- Open GCP-related jobs
- Open Data quality-related jobs
- Open Business Intelligence-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open Data visualization-related jobs
- Open Finance-related jobs
- Open Deep Learning-related jobs
- Open PhD-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open PyTorch-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open CI/CD-related jobs
- Open LLMs-related jobs
- Open Kubernetes-related jobs
- Open Generative AI-related jobs
- Open Data governance-related jobs
- Open Hadoop-related jobs
- Open Airflow-related jobs
- Open Docker-related jobs