SDE II, Search Engine Tech, Search Science and Data Infrastructure

Job summary
The Amazon Search Engine Technologies team builds the search engine that powers Amazon's world wide shopping experience. When a customer searches or browses Amazon website we connect them to the products and services they are looking for.

Right now, we are building next generation capabilities in the Search Engine Infrastructure using the latest advances in Data Engineering, ML, Data Science and Distributed Computing. We have a collaborative startup-like culture and strive to build the right thing right from day one. Our work is at the forefront of business, since we directly impact the Amazon Customer Experience, by delivering behavioral signals to all of Amazon worldwide.

Search Engine Technologies team is responsible for delivering high quality and fresh ML model training data, and providing seamless access to all ML artifacts through managed Federated Data Lake infrastructure. This big-data platform provides the ML training data to Amazon search ranking, matching quality, search economics and also powers live-site features, including search suggestions, query understanding, spelling, search result ranking, and personalization. More than 350 teams across Amazon consume our datasets.

Benefits -

• Flexibility to work from office or home, with the base office in Palo Alto. (2 Days a week in office recommended to maximize overlap with the team.)
• Commuter Friendly - We are located in downtown Palo Alto and right across from the Caltrain station. The location make it great for commute by public transportation.
• Work-Life Balance - We work hard but we know when to stop work and get back to living a wholesome life.
• Access to Major Tech and Scientific Conferences, our Principal Engineers and Scientists constantly advocate for team members to attend conferences, for skill building and networking with peers.
• Open Source Contributors - You get to work with and as an Open Source contributor, ask our Lucene experts how we are developing Lucene into a very custome implementation. Look up PECOS library and the associated training datasets.
• Flexibility to Build Your Career towards a Business Specialization or a Technology Specializations. We believe we need the specialists who are generalists at the core.

As a Software Development Engineer you will own and build:

• Data As A Product: Unblock Business, Science and Engineering Stakeholders by defining new data products, signals, engineered features and insights. Our stakeholders directly influence Search User Experience World Wide using our products. You will work backwards from these customers to build complex distributed computing software that generates enriched signals, features and experimental data sets that drive Search Engagement, Discovery of New Products and Enhanced Experience by serving mission-aware search results.
• Data Organization: You will define how we organize PB of our data on physical storage, and create appropriate indexes to make it easy to access for wide variety of use cases such as ML Training, Analytics, and privacy compliance.
• Data Quality: You will work with our customer to define metrics for data quality that reflect their concerns and relative priority amongst those concerns such as freshness, precision, and completeness. You will use these to define quality metrics for all the intermediate data-sets and take it all the way to define data quality metics for our inputs. You will define SLA for our data input providers for data quality and influence our architecture to gracefully degrade with imperfect data and providing mechanisms for our customers to understand quality of various data-sets so they can use it appropriately. You will define how we do trade-off between various concerns such as freshness, completeness, precision and cost.
• Design & Develop: Lead the design, get your hands dirty and write code, and ultimately deploy big data and machine learning services. These services define the foundation of our search R&D processes, supporting science, product development and production of the worlds largest product search engine.
• Operational Excellence: Obsess over operational excellence, evaluate system performance, security, design system metrics and driving quality improvements
• Obsess over customer needs and satisfaction

In this role, you’ll help establish technical standards and drive the Search Data organization’s overall Data architecture and Engineering practices. You’ll work on the hardest problems, building high quality, architecturally sound systems that are aligned with our business needs and built to handle Amazon's world wide scale. Your expertise is deep and broad; you’re hands on, producing both detailed technical work and high-level architectural designs.

Basic Qualifications

• 1+ years of experience contributing to the system design or architecture (architecture, design patterns, reliability and scaling) of new and current systems.
• 2+ years of non-internship professional software development experience
• Programming experience with at least one software programming language.

:

• Software development and design.
• Writing production code using Java, Scala, and Python

At least 2 years of recent experience in:

• Data Architecture and Data processing for ML and Analytical application
• Defining data and processing interfaces and getting buy-in from across the organization
• Leading delivery of projects requiring work from multiple organizations
• Data transformation/ETL tools and technologies and understanding of related concepts (such as data cataloging and curation, etc.)
• Big data infrastructure such as Hadoop, Spark, Kubernetes,etc.
• Implementation and tuning experience in the Big Data Ecosystem, (such as Hadoop, Spark, Presto, Hive), Database (such as Oracle, MySQL, PostgreSQL, MS SQL Server) and Data Warehouses (such as Redshift, Teradata, Vertica)

Preferred Qualifications

• Graduate degree in computer science or related field (MS or Ph.D.)
• Experience with public cloud infrastructure.
• AWS Certification, e.g. AWS Solutions Architect, Developer, or SysOps Associate/Professional
• Data-driven and “quantitative” mentality. Grounded, detail-oriented, always backs up ideas with facts
• Ability to understand complex application data flows and bridge the gap between technical and business app requirement
• Track record of implementing AWS services in a variety of business such as large enterprises and start-ups

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.