Data Architect - Afternoon Shift
Lahore, Punjab, Pakistan
- Architect, design, automate and maintain optimal data pipeline architecture.
- Developing and implementing an overall organizational data strategy that is in line with business processes. The strategy includes data model designs, database development standards, implementation and management of data warehouses and data analytics systems.
- Identifying data sources, both internal and external, and working out a plan for data management that is aligned with organizational data strategy.
- Managing end-to-end data architecture, from selecting the platform, designing the technical architecture, and developing the application to finally testing and implementing the proposed solution.
- Integrating technical functionality, ensuring data accessibility, accuracy, and security.
- Planning and execution of big data solutions using Hadoop, Spark and HDFS environments. Entail the complete lifecycle management of Hadoop and data management and automation on Spark.
- Architect and build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Conducting a continuous audit of data management system performance, refine whenever required, and report immediately any breach or loopholes to the stakeholders.
- Coordinating and collaborating with cross-functional teams, stakeholders, and vendors for the smooth functioning of the enterprise data system.
- Participate in any other initiatives running under the umbrella of Engineering like training, talks, estimates in Data Engineering domain.
- Influence data engineering best practices within the team.
- Mentor and guide data and information engineers within the company.
- Bachelor’s or master’s degree in Computer Sciences, Engineering or related field
- 10+ experience in the domain of Data Engineering comprises of ETL, reporting tools and Big Data.
- Proficiency in data modeling and design, including SQL development and database management using SQL and PLSQL.
- Experience in ETL tools such as DBT, Talend and Informatica.
- Experience with streaming tools such as Apache Flink and Apache Kafka is a plus.
- Experience in Big Data technologies includes Hadoop, BigQuery and Bigtable.
- Experience in data management and automation using Apache Spark and Apache Airflow.
- Experience working in cloud computing tools like AWS Athena, AWS Glue and Snowflake.
- Enterprise Application and Data Integration design and implementation (e.g., ETL, messaging, replication and APIs).
- Experience with data visualization and data migration.
- Experience in programming languages like Python or Java to develop applications for data analysis.
- Ability to implement common data management and reporting technologies, as well as the basics of columnar and NoSQL databases, data visualization, unstructured data, and predictive analytics
- Understanding of writing effective and maintainable unit and integration tests for ingestion pipelines and awareness of latest CI/CD and automated testing practices.
- Awareness of Data management best practice (e.g., data quality, metadata management, data lineage).
- Awareness of Machine Learning techniques is nice to have.
- Strong business and communication skills.
- Knowledge of Agile development methodologies.
- Ability to perform comfortably in a fast-paced, deadline-oriented work environment.
- Strong interpersonal skills with the ability to work both independently and as part of a team.
- Experience working in an offshore software development environment is a plus.
- Availability to work in the Afternoon Shift hours (1pm to 10pm)
Tags: Agile Airflow Apache Flink APIs Athena AWS Big Data BigQuery CI/CD Data analysis Data Analytics Data management Data strategy Data visualization Engineering ETL Hadoop HDFS Informatica Kafka Machine Learning NoSQL Pipelines Python Security Snowflake Spark SQL Streaming Talend Testing Unstructured data
Perks/benefits: Career development
Other jobs like this
Technical Lead/ Architect- Data Engineer Data BricksAgile APIs AWS Azure Consulting Databricks Data management +12
Career development Startup environment
Technical Lead/Architect - Data Engineer (AWS Redshift/Glue+Pyspark)Agile APIs AWS Consulting Engineering ETL Informatica +12
Career development Startup environment
Explore more AI/ML/Data Science career opportunities
Find open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general, filtered by job title or popular skill, toolset and products used.
- Open Data Analytics Manager jobs
- Open Data Engineer (Remote) jobs
- Open Computer Vision Engineer jobs
- Open Big Data Engineer jobs
- Open Senior Data Analyst (Bangkok Based, relocation provided) jobs
- Open Machine Learning Scientist jobs
- Open Data Engineer II jobs
- Open Research Scientist, Computer Vision jobs
- Open Associate Data Analyst- Customer Experience Group | Bangkok-based jobs
- Open Autonomous Vehicle System Test Specialist jobs
- Open Junior Data Engineer jobs
- Open Data Scientist (Remote) jobs
- Open Marketing Data Analyst jobs
- Open Lead Data Analyst jobs
- Open Data Engineering Lead jobs
- Open Data Analyst (Remote) jobs
- Open Sr. Data Analyst jobs
- Open Head of Data Science jobs
- Open Research Scientist, NLP jobs
- Open Senior Data Architect jobs
- Open Senior Marketing Data Analyst jobs
- Open Junior Data Analyst jobs
- Open Data Scientist II jobs
- Open Senior Data Engineer (Remote) jobs
- Open Data Engineer - Remote jobs
- Open PhD-related jobs
- Open Scala-related jobs
- Open Data visualization-related jobs
- Open TensorFlow-related jobs
- Open Looker-related jobs
- Open APIs-related jobs
- Open Excel-related jobs
- Open Snowflake-related jobs
- Open Business Intelligence-related jobs
- Open Redshift-related jobs
- Open Streaming-related jobs
- Open Hadoop-related jobs
- Open Azure-related jobs
- Open PyTorch-related jobs
- Open Economics-related jobs
- Open Docker-related jobs
- Open Kafka-related jobs
- Open Kubernetes-related jobs
- Open Git-related jobs
- Open NLP-related jobs
- Open Power BI-related jobs
- Open Consulting-related jobs
- Open Pandas-related jobs
- Open BigQuery-related jobs
- Open Data management-related jobs