Senior Data Engineer (New York)

About BrightTALK - discover your journey with BrightTALK here!BrightTALK brings professionals and businesses together to learn and grow. Over 10 million professionals engage with 100,000 free talks and 1,000 online summits to discover new technologies, learn from trusted experts and grow their careers. Thousands of businesses acquire and inspire their ideal audience with BrightTALK's AI-powered content and demand marketing platform. 
We are looking for a talented and passionate Sr. Data Engineer / Architect  to join BrightTALK’s Data team  based in the US East coast.The Data team builds and maintains the infrastructure responsible for consuming, exposing and creating product and product-derived (augmented) datasets. This team owns the pipelines that transport and process database data from all of BrightTALK’s product surfaces. The team is responsible for building and operating the infrastructure and services that ensure data accuracy

Responibilities

• Building and maintain highly reliable data services to integrate with dozens of consumers.
• Creating ETL or ELT pipelines that transform and process structured and unstructured data in real-time.
• Designing data models for optimal storage and retrieval to support high latency.
• Deploying and monitoring large database clusters that are performant and highly-available.
• Working cross-functionally with data scientists, backend engineers, and product managers to design and implement new data models to support the product.
• Developing your skills through exceptional training as well as frequent coaching and mentoring from colleagues.
• Some upcoming technical challenges include scaling our data ingestion pipelines across a growing number of AWS and data center based data sources, reducing the latency of our product data ingestion pipelines through improving our batch jobs, and extending our data lake architecture for the growing ecosystem of data ingestion and creation tools.

Skils

• Bachelor's degree (or equivalent) in Computer Science or related field.
• 5+ years of experience building real-time and distributed architecture, from whiteboard to production
• Strong programming skills in Python, SQL.Versatility.
• Experience across the entire spectrum of data engineering, including:
• Data stores (e.g., AWS RDS, AWS Athena, AWS Aurora, AWS Redshift).
• Data pipeline and workflow orchestration tools (e.g., Azkaban, Airflow).
• Data processing technologies (e.g., Spark, Pentaho).
• Deployment and monitoring batch and realtime integrating with large database clusters in public cloud platforms.
• Adaptable. Goals can change fast. You anticipate and react quickly.
• Autonomous. You own what you work on. You move fast and get things done.
• Excellent communication. You will need communicate complex ideas effectively to both technical and non-technical audiences, and both verbally and in writing.
• Collaborative. You must work collaboratively in a cross-functional team and with people at all levels in an organization.
• Industry experience building and productionizing innovative end-to-end Machine Learning systems is a plus.

Expertise/ Qualifications

• Strong Knowledge of Python & SQL.
• Hands on experience with either Pentaho or other Data Integration tools in the market.
• Demonstrable experience of working in a linux environmentUnderstanding of Terraform and use of it for deployment pipelines.
• Experience working with CI/CD tools.

Why we like working at BrightTALK

• Friendly, talented, collaborative and entrepreneurial teams.
• Generous and comprehensive health and dental benefits, including 100% healthcare coverage for employees.
• Generous holiday policy, flexible schedule and 4 weeks of PTO in the U.S. and 5 weeks of PTO in the U.K. in your first year at BrightTALK.
• 401k matching program in the U.S. and pension contribution in the U.K.
• Offices located in top destination cities. 
• Training allowance and regular innovation days.
• Team building and paid volunteer time.

We are dedicated to learning and growing, going after big dreams, taking ownership, remaining open and honest while recognizing we are in it together. We are committed to hiring individuals with a diversity of perspectives, backgrounds, and beliefs who can use their passion to help us build a world changing audience and thrive along the way. Join BrightTALK and experience a collaborative and engaged team who works together to provide learning and growth opportunities for businesses and professionals. 
BrightTALK is an Equal Opportunity Employer and does not discriminate on the basis of ethnicity, gender, sexual orientation, gender identity or expression, religion, disability, national origin, protected veteran status, age, or any other status protected by applicable national, federal, state, or local law.