Senior Data Engineer
Bengaluru, India
StockX
Buy and sell the hottest sneakers including Adidas Yeezy and Retro Jordans, Supreme streetwear, trading cards, collectibles, designer handbags and luxury watches.Help empower our global customers to connect to culture through their passions.
Technology at StockX
Our Technology Team is on a mission to build the next generation e-commerce platform for the next generation customer. We build world-class, innovative experiences and products that give our users access to the world’s most-coveted products and unlock economic opportunity by turning reselling into a business for anyone. Our team uses cutting edge technologies that handle massive scale globally. We’re an internet-native, cloud-native company from day 1 - you won’t find legacy technology here. If you’re a curious leader who loves solving problems, wearing multiple hats, and learning new things, join us!
About the role
As a Senior Data Engineer, you will be empowered to leverage data to drive amazing customer experiences and business results. You will own the end to end development of data engineering solutions to support analytical needs of the business. The ideal candidate will be passionate about working with disparate datasets and be someone who loves to bring data together to answer business questions at speed. You should have deep expertise in the creation and management of datasets and the proven ability to translate the data into meaningful insights through collaboration with analysts, data scientists and business stakeholders.
What you'll do
Design and build mission critical data pipelines with a highly scalable distributed architecture - including data ingestion (streaming, events and batch), data integration, data curation
Help continually improve ongoing reporting and analysis processes, simplifying self-service support for business stakeholders
Build and support reusable framework to ingest, integration and provision data
Automation of end to end data pipeline with metadata, data quality checks and audit
Build and support a big data platform on the cloud
Define and implement automation of jobs and testing
Optimize the data pipeline to support ML workloads and use cases
Support mission critical applications and near real time data needs from the data platform
Capture and publish metadata and new data to subscribed users
Work collaboratively with business analysts, product managers, data scientists as well as business partners and actively participate in design thinking session
Participate in design and code reviews
Motivate, coach, and serve as a role model and mentor for other development team associates/members that leverage the platform
About you
Minimum of 5 years’ experience in data warehouse / data lake technical architecture
3+ years of experience in using programming languages (Python / Scala / Java / C#) to build data pipelines
Minimum 3 years of Big Data and Big Data tools in one or more of the following: Batch Processing (e.g. Hadoop distributions, Spark), Real time processing (e.g. Kafka, Flink/Spark Streaming)
Minimum of 2 years' experience with AWS or engineering in other cloud environments
Experience with Database Architecture/Schema design
Strong familiarity with batch processing and workflow tools such as AirFlow, NiFi
Ability to work independently with business partners and management to understand their needs and exceed expectations in delivering tools/solutions
Strong interpersonal, verbal and written communication skills and ability to present complex technical/analytical concepts to executive audience
Strong business mindset with customer obsession; ability to collaborate with business partners to identify needs and opportunities for improved data management and delivery
Experience providing technical leadership and mentoring other engineers for best practices on data engineering
Bachelor's degree in Computer Science, or a related technical field
Nice to have:
Masters in Computer Science or related quantitative field
StockX is the premier current culture platform for buying and selling authentic, new, sought-after products. Our powerful marketplace connects buyers and sellers for sneakers, apparel, accessories, electronics, collectibles and trading cards around the world. We provide millions of global customers with unprecedented access and market visibility powered by real-time data, allowing them to transact based on true market value. Launched in 2016 in Detroit, Michigan, StockX now employs more than 1,500 people in offices and authentication centers in 11 countries. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. This job description is intended to convey information essential to understanding the scope of the job and the general nature and level of work performed by job holders within this job. However, this job description is not intended to be an exhaustive list of qualifications, skills, efforts, duties, responsibilities or working conditions associated with the position. StockX reserves the right to amend this job description at any time.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow Architecture AWS Big Data Computer Science Data management Data pipelines Data quality Data warehouse E-commerce Engineering Flink Hadoop Kafka Machine Learning Pipelines Python Scala Spark Streaming Testing
Perks/benefits: Career development Team events
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Research Scientist jobs
- Open Data Science Manager jobs
- Open Junior Data Analyst jobs
- Open Business Data Analyst jobs
- Open Data Scientist II jobs
- Open Principal Data Scientist jobs
- Open Sr Data Engineer jobs
- Open BI Analyst jobs
- Open Business Intelligence Engineer jobs
- Open Sr. Data Scientist jobs
- Open Data Science Intern jobs
- Open Senior Business Intelligence Analyst jobs
- Open Software Engineer, Machine Learning jobs
- Open Lead Data Analyst jobs
- Open Azure Data Engineer jobs
- Open Junior Data Scientist jobs
- Open MLOps Engineer jobs
- Open Manager, Data Engineering jobs
- Open Marketing Data Analyst jobs
- Open Data Analytics Engineer jobs
- Open Data Engineer III jobs
- Open Data Engineering Manager jobs
- Open Junior Data Engineer jobs
- Open Product Data Analyst jobs
- Open Data Analyst II jobs
- Open Data quality-related jobs
- Open Power BI-related jobs
- Open Tableau-related jobs
- Open Excel-related jobs
- Open ML models-related jobs
- Open Data pipelines-related jobs
- Open APIs-related jobs
- Open PhD-related jobs
- Open PyTorch-related jobs
- Open Finance-related jobs
- Open LLMs-related jobs
- Open Deep Learning-related jobs
- Open TensorFlow-related jobs
- Open Data visualization-related jobs
- Open Consulting-related jobs
- Open Business Intelligence-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open NLP-related jobs
- Open Data governance-related jobs
- Open DevOps-related jobs
- Open Kubernetes-related jobs
- Open Git-related jobs
- Open Docker-related jobs
- Open Hadoop-related jobs