Sr Data Engineer (Python + BigData : HDFS/ Hive/ Spark/ Nifi/ Kafka) 4+ years

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

Visa Corporate IT (CIT) group has embarked on a major transformation journey. We are transforming into a true engineering organization that puts customer experience and end-user support first. As part of this strategy, we are consolidating several of our Operational functions into one area focused on providing world-class operational support to our End users. Visa's Finance Engineering, Applications, and Analytics team is looking for curious, creative, and technology experts to join our growing team. We focus on delivering and supporting integrated platforms that enable Visa's finance applications, which Visa business partners use to service Visa clients globally. We also develop analytics and reporting by using revenue data. 

The successful candidate is expected to be an expert in bigdata technologies. The selected candidate will be fully responsible for both service delivery and operational excellence of services built using Hadoop technologies.

RESPONSIBILITIES

• Implement scalable architectural models for data processing and storage.
• Build a framework for data ingestion from multiple heterogeneous sources in batch & real-time mode.
• Build a framework for data analytics, search, and aggregation.
• Optimize existing data processing pipelines,
• Data Integration, Processing & Governance.
• Analytics & Visualizations.

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office two days a week, Tuesdays and Wednesdays with a general guidepost of being in the office 50% of the time based on business needs.

Qualifications

Basic Qualifications

2+ years of relevant work experience and a Bachelors degree, OR 5+ years of relevant work experience

Preferred Qualifications

• Bachelor’s degree in Computer Science with 5+ years of relevant experience.
• Expert-level skills in writing and optimizing complex SQL.
• Knowledge of data warehousing concepts.
• Experience in data mining, profiling, and analysis.
• Experience with complex data modeling, ETL design, and using large databases in a business environment.
• Proficiency with Linux command line and systems administration.
• Experience with languages Python and Java.
• Experience with Big Data technologies such as HDFS/Hive/Spark/Nifi/Kafka.
• Proven ability to develop unconventional solutions. Sees opportunities to innovate and leads the way.
• Provide right and faster solution for any issues.
• Excellent verbal and written communication. Proven interpersonal skills and ability to convey key insights from complex analyses in summarized business terms. Ability to effectively communicate with technical teams.
• Ability to work with shifting deadlines in a fast-paced environment.
• Authoritative in ETL optimization, designing, coding, and tuning big data processes using Apache Spark or similar technologies.
• Experience with building data pipelines and applications to stream and process datasets at low latencies.
• Demonstrate efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
• Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, know how to optimize the distribution, partitioning, and MPP of high-level data structures.
• Knowledge of Engineering and Operational Excellence using standard methodologies.

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.