Data Architect - Afternoon Shift
Lahore, Punjab, Pakistan
Applications have closed
10Pearls
10Pearls | The leading IT, Software, Web, App, and Emerging Technologies Services & Solutions | Enabling & Transforming Digitally Fortune 500 Clients Worldwide- Architect, design, automate and maintain optimal data pipeline architecture.
- Developing and implementing an overall organizational data strategy that is in line with business processes. The strategy includes data model designs, database development standards, implementation and management of data warehouses and data analytics systems.
- Identifying data sources, both internal and external, and working out a plan for data management that is aligned with organizational data strategy.
- Managing end-to-end data architecture, from selecting the platform, designing the technical architecture, and developing the application to finally testing and implementing the proposed solution.
- Integrating technical functionality, ensuring data accessibility, accuracy, and security.
- Planning and execution of big data solutions using Hadoop, Spark and HDFS environments. Entail the complete lifecycle management of Hadoop and data management and automation on Spark.
- Architect and build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Conducting a continuous audit of data management system performance, refine whenever required, and report immediately any breach or loopholes to the stakeholders.
- Coordinating and collaborating with cross-functional teams, stakeholders, and vendors for the smooth functioning of the enterprise data system.
- Participate in any other initiatives running under the umbrella of Engineering like training, talks, estimates in Data Engineering domain.
- Influence data engineering best practices within the team.
- Mentor and guide data and information engineers within the company.
Requirements
- Bachelor’s or master’s degree in Computer Sciences, Engineering or related field
- 10+ experience in the domain of Data Engineering comprises of ETL, reporting tools and Big Data.
- Proficiency in data modeling and design, including SQL development and database management using SQL and PLSQL.
- Experience in ETL tools such as DBT, Talend and Informatica.
- Experience with streaming tools such as Apache Flink and Apache Kafka is a plus.
- Experience in Big Data technologies includes Hadoop, BigQuery and Bigtable.
- Experience in data management and automation using Apache Spark and Apache Airflow.
- Experience working in cloud computing tools like AWS Athena, AWS Glue and Snowflake.
- Enterprise Application and Data Integration design and implementation (e.g., ETL, messaging, replication and APIs).
- Experience with data visualization and data migration.
- Experience in programming languages like Python or Java to develop applications for data analysis.
- Ability to implement common data management and reporting technologies, as well as the basics of columnar and NoSQL databases, data visualization, unstructured data, and predictive analytics
- Understanding of writing effective and maintainable unit and integration tests for ingestion pipelines and awareness of latest CI/CD and automated testing practices.
- Awareness of Data management best practice (e.g., data quality, metadata management, data lineage).
- Awareness of Machine Learning techniques is nice to have.
- Strong business and communication skills.
- Knowledge of Agile development methodologies.
- Ability to perform comfortably in a fast-paced, deadline-oriented work environment.
- Strong interpersonal skills with the ability to work both independently and as part of a team.
- Experience working in an offshore software development environment is a plus.
- Availability to work in the Afternoon Shift hours (1pm to 10pm)
Tags: Agile Airflow APIs Architecture Athena AWS Big Data BigQuery Bigtable CI/CD Data analysis Data Analytics Data management Data quality Data strategy Data visualization Engineering ETL Flink Hadoop HDFS Informatica Kafka Machine Learning NoSQL Pipelines Python Security Snowflake Spark SQL Streaming Talend Testing Unstructured data
Perks/benefits: Career development
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Marketing Data Analyst jobs
- Open MLOps Engineer jobs
- Open AI Engineer jobs
- Open Junior Data Scientist jobs
- Open Data Engineer II jobs
- Open Senior Data Architect jobs
- Open Sr Data Engineer jobs
- Open Data Analytics Engineer jobs
- Open Power BI Developer jobs
- Open Senior Business Intelligence Analyst jobs
- Open Principal Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Product Data Analyst jobs
- Open Business Data Analyst jobs
- Open Data Manager jobs
- Open Data Quality Analyst jobs
- Open Sr. Data Scientist jobs
- Open Data Scientist II jobs
- Open Big Data Engineer jobs
- Open Business Intelligence Developer jobs
- Open Data Analyst Intern jobs
- Open Principal Data Scientist jobs
- Open ETL Developer jobs
- Open Azure Data Engineer jobs
- Open Data Product Manager jobs
- Open Business Intelligence-related jobs
- Open Data quality-related jobs
- Open Privacy-related jobs
- Open Data management-related jobs
- Open GCP-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Finance-related jobs
- Open Data visualization-related jobs
- Open Deep Learning-related jobs
- Open APIs-related jobs
- Open PyTorch-related jobs
- Open PhD-related jobs
- Open TensorFlow-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open NLP-related jobs
- Open Data governance-related jobs
- Open Data warehouse-related jobs
- Open Airflow-related jobs
- Open Databricks-related jobs
- Open Hadoop-related jobs
- Open LLMs-related jobs
- Open DevOps-related jobs
- Open CI/CD-related jobs