Data Engineer-delivery

Company Description

We help the world see new possibilities and inspire change for better tomorrows. Our analytic solutions bridge content, data, and analytics to help business, people, and society become stronger, more resilient, and sustainable.

**Job Description**:
In this role you will be responsible for expanding and optimizing our data pipeline architecture. The Data Engineer will support Product Delivery, Engineering, Analytics, and all of our data scientists on various initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.

ESSENTIAL FUNCTIONS:

- Help architect and maintain our code base for ETL pipelines, large batch processing, and streaming systems
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Create and maintain toolsets to help manage our enterprise data pipelines
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Ensure our code meets quality standards and architectural requirements
- Help build out the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
- Comfortable writing stories and associated acceptance criteria for agile/scrum workflow
- Help keep our data separated and secure across multiple data centers and AWS regions.

**Qualifications**:

- Experience working under agile/scrum and the major pieces of that framework (epics, stories, sprint planning and retros),
- Intermediate/Advanced knowledge of Python
- Experience with and knowledge of large batch data processing
- Experience with SQL and NoSQL databases, and an understanding of key concepts (Foreign Keys, Indices, etc)
- Experience with Big Data technologies, preferably PySpark/ Spark, Airflow, or Pandas
- Experience seeing the full end to end SDLC of a system to production.
- Basic understanding of bash scripting, and the basics of Linux operating systems.
- Understanding of Microservice architecture and preferably familiarity with 12 Factor architecture.
- Experience with CI/CD, such as Jenkins + Gitlab
- 1-2 years experience developing and deploying solutions on AWS and familiarity with AWS' ecosystem of services
- Experience with Docker or Kubernetes a plus
- Experience with stream-processing systems a plus
- Experience with BDD/TDD is a plus

Additional Information

Our company of rapidly-growing data, technology and analytics leaders is looking for a to join our team that has recently been recognized as:

- Crain’s Best Places to Work Company
- Crain’s Best-Paying Company
- BuiltIn Best Places to Work
- Inc. 5000 Fastest Growing Companies in the US
- AdWeek Top 20 Fastest Growing Solution Providers
- Martech Breakthrough Award Winner for Best Contact Database Solution

Verisk (Nasdaq:VRSK) provides predictive analytics and decision-support solutions to customers in the insurance, energy and specialized markets, and financial services industries. More than 70 percent of the FORTUNE 100 relies on the company’s advanced technologies to manage risks, make better decisions and improve operating efficiency. Celebrating its 50th anniversary, the company continues to make the world better, safer and stronger, and fosters an inclusive and diverse culture where all team members feel they belong.

With more than 100 offices in nearly 35 our global team uses AI, machine learning, automation, and other emerging technologies to collect and analyze billions of records. We provide advanced decision-support to prevent credit, lending, and cyber risks. In addition, we monitor and advise companies on complex global matters such as climate change, catastrophes, and geopolitical issues.

But why we do our work is what sets us apart. It stems from a commitment to making the world better, safer and stronger.

It’s the reason Verisk is part of the UN Global Compact sustainability initiative. It’s why we made a commitment to balancing 100 percent of our carbon emissions. It’s the aim of our “returnship” program for experienced professionals rejoining the workforce after time away. And, it’s what drives our annual Innovation Day, where we identify our next first-to-market innovations to solve our customers’ problems.

At its core, Verisk uses data to minimize risk and maximize value. But far bigger, is why we do what we do.

At Verisk you can build an exciting career with meaningful work; create positive and lasting impact on business; and find the support, coaching, and training you need to advance your career. We have received the Great Place to Work® Certification for the fifth consecutive year. We’ve been recognized by Forbes as a World’s Best Employer and a Best Employer for Women, testaments to our culture of engagement and the value we place on an inclusive and diverse wor