Senior Data Engineer
As a Senior Data Engineer, you will play an instrumental role in advancing our core machine learning solution. Your technical expertise and leadership skills will be leveraged to drive the development and deployment of high-quality, scalable data pipelines and products. You will work alongside scientists and engineers in a collaborative environment, contributing features and data pipelines to power our core recommender systems and our DaVinci Personalization product. This is an opportunity to work end-to-end on a large-scale machine-learning system that touches millions of customers, and a chance to continuously learn and help improve our solution as the field evolves.
This role will be reporting to the Director of Engineering (AI).
Responsibilities
- Implement production data products and pipelines that are scalable, reliable, and of high quality.
- Build, maintain, and optimize our machine learning Data Lake.
- Continuously improve data infrastructure for greater scalability.
- Support the data needs of ML Engineers and Scientists for machine learning model development.
- Release features that deliver measurable and tangible business value.
Requirements
- 4+ years of data engineering experience
- Experience with large-scale data processing frameworks (we use PySpark, SQL)
- Expertise in Spark DataFrame API
- Experience with event stream data (we use Kafka)
- Strong software development skills in Python (unit testing, git, code review, CI/CD)
- Experience with cloud computing platforms and cluster configuration, optimization, and scaling (GCP)
- Experience in data storage formats (we use Parquet, Delta Lake)
- Ability to collaborate with technical partners – you’ll be working closely with other teams to determine requirements for your work and to make design decisions that affect our stack
- Enjoys working in a fast-paced, goal-driven environment
The base pay range for this position is $180,000-$210,000 USD/year, which can include additional on-target commission pay/bonus. The base pay offered may vary depending on job-related knowledge, skills, and experience. Stock options and other incentive pay may be provided as part of the compensation package, in addition to a full range of medical, financial, and/or other benefits, depending on the position ultimately offered.
Studies have shown that women, communities of color, and historically underrepresented people are less likely to apply to jobs unless they meet every single qualification. We are committed to building a diverse and inclusive culture where all Inkers can thrive. If you’re excited about the role but don’t meet all of the abovementioned qualifications, we encourage you to apply. Our differences bring a breadth of knowledge and perspectives that makes us collectively stronger.
We welcome and employ people regardless of race, color, gender identity or expression, religion, genetic information, parental or pregnancy status, national origin, sexual orientation, age, citizenship, marital status, ethnicity, family or marital status, physical and mental ability, political affiliation, disability, Veteran status, or other protected characteristics. We are proud to be an equal opportunity employer.