Deep Learning Inference Edge SDE

East Palo Alto, California, USA

Full Time
Amazon.com logo
Amazon.com
Apply now Apply later

Posted 1 month ago

At AWS AI, we want to make it easy for our customers to deploy machine learning models on any endpoint in the cloud or at the edge. Just as SageMaker provides a complete set of services to simplify the task of building and training a model, Neo provides an inference engine that is designed to run any machine learning model on any hardware. Running machine learning inference on edge devices reduces latency, conserves bandwidth, improves privacy and enables smarter applications, and is a rapidly growing area as smart devices proliferate consumer and industrial applications.

Neo optimizes machine learning models to perform at up to twice the speed of the original framework with no loss in accuracy. Upload a pre-trained model built with MXNet, TensorFlow, PyTorch, or XGBoost to your S3 bucket, choose your target hardware platform from Intel, NVIDIA, or ARM, and with a single API call, SageMaker Neo optimizes the model, converts it into an executable module, and returns it to your S3 bucket. The free open source Neo runtime uses less than 100th of the space of the framework to run the model on the target hardware.

The SageMaker Neo Edge team is building new technology to deliver the most efficient inference on edge devices, coupled with ease of managing these devices. We are hiring well-rounded applied scientists and software developers with backgrounds in machine learning, embedded systems, compilers, and AI accelerators. If you have worked on running high performance computations on embedded devices, done performance tuning, developed software stacks on devices with limited compute and memory resources, you will enjoy working on the breadth of ML applications that we optimize.


You will help develop the runtime for machine learning, work with machine learning model compilers, and enable device and model management. You will engage with silicon vendors and device makers on enriching their on-device and device management service needs. The role offers an extremely broad set of opportunities to work as a full stack SDE with exposure to multiple AI applications, ML frameworks, ML models, compilers, embedded software, service software, and various AI hardware including ARM, Intel, AWS Inferentia, and NVidia.

Join the Amazon SageMaker Neo Edge team to help AWS customers deploy machine learning models on edge devices at scale in production. Work on an open source industry-standard compiler and runtime for machine learning that is already deployed on over 20 million devices.

Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation / Age

Basic Qualifications


· Master's Degree in Computer Science or Engineering
· 1+ years of software development experience in an embedded system platform for a shipping product. Hands on experience with multi-threaded software stacks.
· 1+ years C/C++ experience
· 1+ years of software development experience in system security -- encryption, access control, auditing, trusted execution, SELinux, container security, or related technologies

Preferred Qualifications



· 1+ years technical leadership role for ground up design and delivery of key on-device software modules, including writing documentation, designing APIs, doing architecture reviews, and leading more junior engineers.
· Familiarity with a Machine Learning framework such as TensorFlow, PyTorch, or MxNet.
· Experience with running inference runtimes on embedded platforms
· Experience making optimization trade-offs for Machine Learning operations on various processors

Job tags: AI AWS Deep Learning Engineering Industrial Machine Learning ML MXNet Open Source PyTorch Security TensorFlow XGBoost