Data Engineer

Bangalore, Karnataka, IN

Full Time
IBM logo
IBM
Apply now Apply later

Posted 2 weeks ago


Introduction
At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. To lead in this new era of technology and solve some of the world's most challenging problems.


Your Role and Responsibilities
Who You Are
  • As a Data Engineer, you play a vital role in building the right infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
  • Expert in setting up effective pipelines to capture data from multiple sources into the enterprise centric storage.
  • Comfortable in building effective analytical tools that utilize the data pipeline to provide actionable insights into data synchronization, reporting, operational efficiency and related areas.
  • Possess strong knowledge in designing database models to store structured & unstructured data efficiently and in creating effective data tools for analytics experts.
  • Knowledge in technologies like Hadoop, Spark, Kafka, Scala, Python, etc.
  • Knowledge in relational model databases (like DB2, MySQL, Oracle, …) and no-SQL databases (MongoDB, Elastic Search, …).
  • Knowledge on enterprise data lakes, data analytics, reporting, in-memory data handling, enterprise integration tools, etc.
  • Good understanding of industry best practices for data governance and security.
  • Good communication skills and fluent in English.

What You Will Do
  • Work with stakeholders including the product owner, data and design teams to assist with data-related technical issues and support their data infrastructure needs.
  • Create and maintain optimal data pipeline architecture.
  • Identify, design, and implement process improvements aimed at automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Assemble large, complex data sets including legacy structured data warehouse that meet functional / non-functional business requirements.
  • Collaborate with DevOps team to develop Continuous Integration/Continuous Delivery pipelines using containerization technologies.
  • Solve Big Data and Distributed Data Streaming problems using latest technologies.
  • Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Manipulate, process and extract value from large disconnected datasets.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management
  • Work on State-of-the-Art cloud technologies provided by IBM Public Cloud, RedHat, AWS & others.
  • Be part of open, transparent agile teams who always thrive for continuous learning and contribute towards continuous improvement.
How We’ll Help You Grow
  • Opportunities to work closely with our industry, technical and functional leads.
  • Excellent platform to drive end-to-end business and process transformation.
  • Access to IBM’s business and management related education
  • Constructive mentorship and career progression options



Required Technical and Professional Expertise
  • Hadoop, Spark, Kafka, Scala, Python, DB2, MySQL, Oracle, no-SQL databases like MongoDB, Elastic Search



Preferred Technical and Professional Expertise
  • Enterprise data lakes, data analytics, reporting, in-memory data handling, enterprise integration tools, etc.
  • Good communication skills



About Business Unit
We at IBM Chief Information Office (CIO) are a dynamic group of Business, Strategy and Technology professionals - a specific source of market-leading Industry Consulting, Application and Business process delivery following Agile values and principles. CIO is at the forefront of Digital Reinvention of key applications used within IBM providing value-led and asset-powered end to end solutions.

CIO mission is to create a productive environment for everyone at IBM. We do this by leading with Design to drive simplicity and ease of use, Engineering the systems that run the business, and Innovating to transform the business. Key focus areas are to secure the Enterprise in network security, endpoint security and data security
Transform IBM and improve collaboration and practice a culture of agile way of working.

Within the CIO, we represent the Analytic Solutions area of the Sales & Marketing Systems. Our mission is to provide business insights to our partners in sales and marketing to win in the market place through analytics solutions delivered using latest technology and based on real time, consolidated and cloud-based data.


Your Life @ IBM
We at IBM Chief Information Office (CIO) are a dynamic group of Business, Strategy and Technology professionals - a specific source of market-leading Industry Consulting, Application and Business process delivery following Agile values and principles. CIO is at the forefront of Digital Reinvention of key applications used within IBM providing value-led and asset-powered end to end solutions.

CIO mission is to create a productive environment for everyone at IBM. We do this by leading with Design to drive simplicity and ease of use, Engineering the systems that run the business, and Innovating to transform the business. Key focus areas are to secure the Enterprise in network security, endpoint security and data security
Transform IBM and improve collaboration and practice a culture of agile way of working.

Within the CIO, we do represent the Analytic Solutions area of the Sales & Marketing Systems. Our Mission is to provide business insights to our partners in sales and marketing to win in the market-place through analytics solutions delivered using latest technology and based on real time, consolidated and cloud based data.


About IBM
IBM’s greatest invention is the IBMer. We believe that progress is made through progressive thinking, progressive leadership, progressive policy and progressive action. IBMers believe that the application of intelligence, reason and science can improve business, society and the human condition. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 380,000 IBMers serving clients in 170 countries.


Location Statement
For additional information about location requirements, please discuss with the recruiter following submission of your application.


Being You @ IBM
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.









Job tags: AWS Big Data Consulting Data Analytics Engineering Hadoop Kafka Marketing MongoDB MySQL Oracle Python Scala Security Spark SQL