Senior Data Engineer
|Location||New York, NY or Remote (North America, U.S. Work Permit)|
|Starts||As soon as possible|
The Senior Data Engineer will build meaningful analytics that will inform companies of security risk. The engineer will be working closely with our Data Science team, implementing algorithms and managing the analytic pipeline. The ideal candidate will have experience processing and querying large amounts of data.
This role requires senior level experience with Spark, SQL and Scala. Our interview process will include live coding using these technologies.
- Manage the analytic pipeline using Spark, Hadoop, etc
- Leverage cutting-edge technologies to support new and existing and services and processes.
- Quickly and efficiently design and implement in an agile environment
- Work with other team members to implement consistent architecture
- Drive projects through all stages of development
- Actively share knowledge and responsibility with other team members and teams
- Improve the effective output of the engineering team by managing quality, and identifying inconsistencies.
Skills and Experience:
- Bachelor’s degree (CS, EE or Math preferred) or equivalent work experience as well as interest in a fast paced, complex environment.
- 5+ years of experience Scala preferred in a commercial environment
- Expert in Spark, experience with the Hadoop ecosystem and similar frameworks
- Expert in SQL
- Familiarity with various tools such as AWS and Docker and an instinct for automation
- Strong understanding of Software Architecture principles and patterns.
- Experience working with 3rd party software and libraries, including open source
- Experience with Postgres
- Quick-thinker who takes ownership and pride in their work
- A commitment and drive for excellence and continual improvement
- A strong sense of adventure, excitement and enthusiasm.
- Excellent systems analytical, problem solving and interpersonal skills