Senior Data Engineer - Contractor

Job Summary:

Square Enix has an internal Google Cloud Platform-based Analytics platform (SGV), which provides our Analytics & Insight team and other groups across the business with a single data lake pooling game telemetry, sales and marketing data, web analytics and other information.

Duties include:

• Optimising, refining and enhancing the data acquisition pipelines
• Working with client teams to ensure robust capture of high-quality data
• Supporting data analysts and other users of the data via training and technical assistance
• Providing staff development for others in the Data Engineering team

This position requires a driven and talented person that can help the team progress.

Requirements

Key Deliverables:

• Improving our Streaming & Batch pipelines to enhance our compliance processes, this includes:
• Ensuring backwards compatibility with currently running services
• Designing how our pipelines would handle data and define the desired destination

• The backup processes handle and write data to same structure as the database

• Pipelines would need to validate regional laws to calculate how to handle the data

• Implementing a Data Lineage solution to track how data is moving through our data lake

• Integrating with new data sources within the business

• Implement a process to ensure the backups are compressed on a regular basis

• Design and develop monitoring solutions to ensure processes are succeeding and data is of sufficient quality and meets our standards

• General day to day bug fixing & monitoring

Key Stakeholders:

Head of Digital Channels, Online Development Director, Director of Analytics & Insight

Knowledge & Experience

Essential:

• High-level of professional experience with cloud-based data engineering platforms ideally Google Cloud Platform (DataFlow, BigQuery, PubSub, GCS)

• Expertise with lambda architecture and other approaches to capture and processing of data at scale to provide real-time analytics capability

• Excellent programming skills in Java & Python is essential, other languages an advantage. We predominantly use Java 11.

• Experience modelling ETLs using Apache Beam.

• Experience writing near real-time ETLs.

• Experience with multiple build tools, preferably gradle

• Expert SQL skills

• Comfortable familiarity working with large data sets

• Excellent problem solving & analytical skills.

• Familiarity with OSX or Linux environment (shell scripting, basic system administration etc).

• Experience with managing a code base and using source control/collaboration tools such as GitHub, Bitbucket or GitLab.
• Familiarity with collaboration and communication tools such as JIRA, Confluence, Slack etc.

Desirable:

• BSc or higher level degree in Computer Science, STEM subject or a similar field of study

• Experience with a variety of systems with aggregation frame works such as Mongo and Elastic Search

• Experience with DAG based workflow management systems, ideally AirFlow

• Interested in statistical methodologies and models.

• Experience with Hadoop technologies.

• Experience writing ETLs in SPARK.

• Knowledge of functional programming languages, such as Scala, Kotlin etc

• Knowledge of Data Protection laws and best practices

• Experience working in a data protection regulated environment would be beneficial

Competencies, Skills & Attributes

Essential:

• Ability to quickly learn and employ new technologies and methodologies.
• Highly numerate.
• Strong documentation skills.
• Ability to articulate and present ideas and information with ease and clarity.
• Ability to work on own initiative and as part of team.

Other:

• Strong interest in technology.
• Ambition to drive self-development
• Excellent attention to detail.
• Ability to work under pressure and to deadlines.
• Follower of industry trends and developments.

Our goal at Square Enix is to hire, retain, develop and promote the best talent, regardless of age, gender, race, religious, belief, sexual orientation or physical ability.

Our pledge to D&I

At Square Enix we believe in the importance of being a diverse and global company, and we stand firmly together against any forms of injustice, intolerance, harassment or discrimination. In our effort to create a truly diverse workforce, we pledge to continue to raise awareness in every step of the employee experience, from recruitment to promotions to ensure equal opportunities for all. One of our goals is to champion diversity in games and at work and work together to inspire real change.

Learning and education around D&I will be a key element for us to continue to grow as an organization. With unconscious bias training, D&I workshops and a variety of initiatives to give our employees the opportunity to be heard and be part of that change to achieve real equality. We need all our efforts to continue to build our culture of inclusion and equality.

We are also proud to partner with UKIE's Raise the Game pledge, BAME in Games and Women in Games, to name a few.

Covid-19 remote working

Square Enix are committed to keeping our employees safe. We are listening closely to government guidelines, and this role will be based remotely until the company sees fit to return to the office. This role will be based in our Blackfriars Studio in London, upon return.