Lead Big Data Engineer

Location: Sydney,New South Wales,Australia

Quantium

Since 2002, Quantium have combined the best of human and artificial intelligence to power possibilities for individuals, organisations and society. Our solutions make sense of what has happened and what will, could or should be done to re-shape industries and societies around the needs of the people they serve.

As one of the world’s fully diversified data science and AI leaders we operate across every sector of the economy and we’re growing fast - with growth comes opportunity! We’re passionate about building out our team of smart, fun, diverse and motivated people.

We combine a team of experts that spans data scientists, actuaries, statisticians, business analysts, strategy consultants, engineers, technologists, programmers, product developers, and futurists – all dedicated to harnessing the power of data to drive transformational outcomes for our clients.

We actively foster a culture where our people can stretch themselves to reach their full potential.
We also know that work has to work for you, and modern life is fast-paced and balance can be tricky. You want to work where you are respected and valued as an individual, not a number. Quantium embraces a flexible and supportive environment dedicated to powering possibilities for our team members, clients and partners.

Role summary

As a Lead Engineer you will be accountable for the enhancement, maintenance and support of the product on which your team is working, within your technology area. You will be responsible for your own hands-on coding, ensuring the quality of your team’s output, representing your team in product-level forums and ensuring your team provides input to and aligns with the overall product road-map.

You will work with Engineers in other technology areas to define the overall technical direction for the product, with product owners and business stakeholders to shape the product's delivery road-map and with support teams to ensure its smooth operation.

You will also be responsible for line management of your team of Engineers, potentially across multiple locations, ensuring that they perform to the expected levels and that their career development is fully supported.

Key responsibilities

• Produce Quality Code
  • Code follows team standards, is structured to ensure readability and maintainability and goes through review smoothly, even for complex changes
  • Designs respect best practices and are favourably reviewed by peers
  • Critical paths through code are covered by appropriate tests
  • High-level designs / architectures align to wider technical strategy, presenting reusable APIs where possible and minimizing system dependencies
  • Data updates are monitored and complete within SLA
• Operate at a High Level of Productivity
  • Estimates are consistently ‘challenging, but realistic’
  • Most tasks are delivered within estimate
  • Complex or larger tasks are delivered autonomously
• Squad Collaboration
  • Sprint goals are consistently achieved
  • Demonstrate commitment to continuous improvement of squad activities
  • The product backlog is consistently well-groomed, with a responsible balance of new features and technical debt mitigation
  • Other Engineers in the Squad feel supported in their development
• People Management
  • Direct reports have meaningful objectives recorded in Quantium's Performance Portal, and understand how those objectives relate to business strategy
  • Direct reports' career aspirations are understood / documented, with action plans in place to move towards those goals
  • Direct reports have regular catch-ups to discuss performance, career development and their on-going happiness / engagement in their role
  • Any performance issues are identified, documented and agreed, with realistic remedial plans in place

Key activities

• Write polished code, aligned to team standards, including appropriate unit / integration tests
• Review code and test cases produced by others, to ensure changes satisfy the associated business requirement, follow best practices, and integrate with the existing code-base
• Provide constructive feedback to other team members on quality of code and test cases
• Collaborate with other Lead / Senior Engineers to produce high-level designs for larger pieces of work
• Validate technical designs and estimates produced by other team members
• Merge reviewed code into release branches, resolving any conflicts that arise, and periodically deploy updates to production and non-production environments
• Troubleshoot production problems and raise / prioritize bug tickets to resolve any issues
• Proactively monitor system health and act to report / resolve any issues
• Provide out of hours support for periodic ETL processes, ensuring SLAs are met
• Work with business stakeholders and other leads to define and estimate new epics
• Contribute to backlog refinement sessions, helping to break down each epic into a collection of smaller user stories that will deliver the overall feature
• Work closely with Product Owners to ensure the product backlog is prioritized to maximize business value and manage technical debt
• Lead work breakdown sessions to define the technical tasks required to implement each user story
• Contribute to sprint planning sessions, ensuring the team takes a 'realistic but challenging' amount of work into each sprint and each team member will be productively occupied
• Contribute to the team’s daily stand-up, highlighting any delays or impediments to progress and proposing mitigation for those issues
• Contribute to sprint review and sprint retro sessions, to maintain a culture of continuous improvement within the team
• Coach / mentor more junior Engineers to support their continuing development
• Set and periodically review delivery and development objectives for direct reports
• Identify each direct reports’ longer-term career objectives and, as far as possible, factor this into work assignments
• Hold fortnightly catch-ups with direct reports to review progress against objectives, assess engagement and give them the opportunity to raise concerns about the product or team
• Work through the annual performance review process for all team members
• Conduct technical interviews as necessary to recruit new Engineers

Experience and education required

• 5+ years' experience developing at scale applications using Scala
• Proven experience manipulating large data-sets using Spark
• Solid foundation in functional programming and data structures
• Values delivering high-quality, peer-reviewed, well-tested code
• Awareness of DevOps functions and appetite to contribute to CI / CD pipelines
• Experience working with source control tools (GIT preferred) with good understanding of branching / merging strategies
• Bachelor’s degree in Computer Science, Information Technology or a related discipline
• Comfortable working in a fast moving, agile development environment
• Excellent problem solving / analytical skills
• Good written / verbal communication skills
• Commercially aware, with the ability to work with a diverse range of stakeholders
• Enthusiasm for coaching and mentoring more junior engineers
• Experience in leading teams, including line management responsibilities

What does success look like?

• New features delivered within estimate, with minimal defects and no material errors
• Larger epics coordinated with business stakeholders and across technology streams
• Recognized as a 'go-to' person for input to high-level designs and estimation
• System availability and data updates complete within SLA
• All user queries and support incidents handled within the team
• Objectives and development plans in place for all team members
• Employee engagement high within the team

We recognize that it takes time to adapt to a new role and to build relationships with key contacts in the business. We expect a successful employee to be on the following trajectory:

• After 1 month you should be:
  • embedded within your squad, engaging easily with other team members and raising constructive feedback on the team's work
  • working autonomously on simple tasks / user stories
  • holding regular catch-ups with team members and be aware of their existing objectives and development plans
• After 3 months, you should be:
  • self-sufficient on all but the most complex tasks / user stories, delivering good quality code in line with other team members' estimations
  • sufficiently aware of the business domain and product road-map to be active in team meetings and contribute towards the product road-map
  • recognized by business stakeholders as a technical authority and able to influence the technical direction of the product
  • shaping team members' objectives and delivery plans in line with their abilities
  • building a network of contacts across Engineering and your business vertical
• After 6 months, you should have:
  • fully effective in your role within the team
  • capable of mentoring / on-boarding new starters into more junior roles

Key business capabilities required

1. Development – Applying technical expertise to construct a solution to meet specific objectives or requirements
2. Fast-Fail / Agile Testing – Ability to try something, get fast feedback, and then rapidly inspect and adapt for great effectiveness
3. Implementation Phase Design – Ability to plan the execution of an application, idea, model, design, specification, standard, algorithm, or policy
4. High Level Design – Ability to define an overview of an entire system, identifying the main components that would be developed for the product/technology and their interfaces
5. Release Management – Ability to oversee the testing, deployment and support of software or product releases

Key People and Leadership capabilities required

1. Self-aware - you leverage diversity across people, tasks, client interactions and projects, taking responsibility for self and other
2. Agile and innovative - you possess strong lateral thinking skills and actively develop these in others
3. Achieve and perform - you have exceptional execution skills and are achievement focused
4. Brand advocate - you anticipate and consider brand and cultural impact in decision making
5. Purposeful and aligned - you are able to set clear, tangible objectives which deliver against our strategy
6. Achievement oriented - you effectively facilitate challenging performance related conversations in a timely and appropriate manner
7. Coach - you demonstrate the ability to coach and develop others while facilitating learning, growth and engagement