Full-stack Data Engineer
State of São Paulo, Brazil - Remote
Xenon7 specializes in integrating Artificial Intelligence (AI) into businesses, offering services that leverage AI to optimize operations, enhance customer experiences, and drive innovation. We emphasize AI as a fundamental tool for business growth and competitive advantage, catering to various industries with a focus on data-driven decision-making and operational efficiency.
We are seeking a versatile Full-Stack Data Engineer with expertise in Python programming, AWS services, DevOps practices, and a solid understanding of both software and data engineering principles. This role is pivotal in building and maintaining scalable data infrastructure, automating data processes, and contributing to the development of software solutions that manage and analyze large data sets.
Responsibilities:
- Design, implement, and manage microservices architecture using AWS cloud services.
- Develop and maintain automated data pipelines, ensuring data accuracy and efficient processing using Airflow DAG.
- Implement DevOps strategies for continuous integration and continuous deployment (CI/CD) to improve software development processes.
- Write high-quality, maintainable code in Python to support data operations.
- Integrate various internal and external data sources (including SharePoint, One Drive, One Note, etc.) to create a unified data platform.
- Build tools for document ingestion, dynamic file previewing, and tabular data extraction.
- Manage graph database systems and support large text corpus management.
- Implement security controls to maintain data integrity and compliance.
- Use Large Language Models for keyword extraction and data analysis to gain insights.
Requirements
- Proven experience with Python programming and AWS services (EC2, RDS, S3, Lambda, etc.).
- Proficient in managing and orchestrating workflows with Airflow.
- Solid understanding of DevOps principles, including experience with CI/CD tools and practices.
- Demonstrated experience with data engineering and familiarity with graph database management.
- Expertise in data processing, corpus management, and keyword extraction.
- Strong problem-solving abilities, attention to detail, and organizational skills.
- Excellent communication and teamwork skills to collaborate with cross-functional teams.
- Minimum overlap of 5 hours with EST time zone during working hours.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow Architecture AWS CI/CD Data analysis DataOps Data pipelines DevOps EC2 Engineering Lambda LLMs Microservices Pipelines Python Security SharePoint
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Data Science Manager jobs
- Open Marketing Data Analyst jobs
- Open MLOps Engineer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Engineer II jobs
- Open Principal Data Engineer jobs
- Open Data Manager jobs
- Open Power BI Developer jobs
- Open Data Scientist II jobs
- Open Sr Data Engineer jobs
- Open Business Data Analyst jobs
- Open Junior Data Scientist jobs
- Open Data Analytics Engineer jobs
- Open Product Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Data Analyst Intern jobs
- Open Sr. Data Scientist jobs
- Open Senior Data Architect jobs
- Open Big Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Principal Data Scientist jobs
- Open Azure Data Engineer jobs
- Open Data Quality Analyst jobs
- Open Junior Data Engineer jobs
- Open Research Scientist jobs
- Open Data quality-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open Business Intelligence-related jobs
- Open ML models-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Deep Learning-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open PyTorch-related jobs
- Open APIs-related jobs
- Open TensorFlow-related jobs
- Open NLP-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open LLMs-related jobs
- Open Generative AI-related jobs
- Open CI/CD-related jobs
- Open Kubernetes-related jobs
- Open Hadoop-related jobs
- Open Data governance-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs