Data Engineer
- Do you dream about Data and talk to your friends in SQL?
- Do you want to inspire and be inspired by working with the best and brightest?
- You’re friendly, positive, professional, and fun to work with!
- You’re a creative thinker with excellent problem solving and decision making ability.
- You’re proactive, self starting, organized, and willing to take on difficult problems.
- You have excellent communication skills, both written and verbal.
- You’re self motivated, energetic, and passionate.
You’ll be the genius who understands data at Dropbox, knows where to find it and manages the process to make that data useful for Growth and Monetization at Dropbox. You love thinking about the ways the business can consume this data and then figuring out how to build it. On a typical day you may be consulted on the information architecture of our subscriptions / invoicing systems and help design the event collection infrastructure. You will also be responsible for understanding and curating Gold datasets associated to all Dropbox customers that will be used for marketing, segmentation and several other analytics and operational use cases. You’ll build the data models and ETL processes to provide this data for business use. You've got some practical experience working with large datasets. You are interested in reporting platforms and data visualization.
You may be interested in machine learning or statistics or one of several similar fields. But the most important factor is you have a strong foundation in test driven development and building scalable data pipelines. As part of the Dropbox’s FDE team, you'll own a problem end-to-end, so those skills will come in handy not just to collect, extract and clean the data, but also to understand the systems that generated it, and automate your processes to produce reliable and trustworthy data. On an on-going basis, you'll be responsible for improving the data by adding new sources, coding business rules and producing new metrics that support the business.
As a data engineer, you have experience spanning traditional DW and ETL architectures and big data ecosystems like Databricks / Snowflake / EMR / Airflow. You’ve probably been in the industry an engineer and have developed a passion for the data that drives businesses. You'll need to be technical at heart, comfortable with the idea of extending systems by writing code, rather than just relying on in-built functionality.
Our Engineering Career Framework is viewable by anyone outside the company and describes what’s expected for our engineers at each of our career levels. Check out our blog post on this topic and more here.
Responsibilities- Build and manage data pipeline generating P0 (most critical) metrics for Dropbox
- Pay meticulous attention to end-to-end data quality, validation, and consistency
- Solve our most challenging data integration problems, utilizing optimal ETL patterns, frameworks, query techniques, sourcing from structured and unstructured data sources
- Challenge status quo and adopt industry best practices
- Work closely with Dropbox business and engineering teams and anticipate how they'd like to use the curated data sets
- Excellent communication skills and comfortable with Agile software development methodologies
- Collaborate with cross functional teams, product managers, and data scientists to understand data needs, representing key data insights in a meaningful way
- Define and manage SLA for all data sets in allocated areas of ownership
- Build cost optimized and efficient data pipelines that are easy to govern and manage
- Strong ownership to solve 0-1 problems with minimal guidance and being comfortable with ambiguities
Many teams at Dropbox run Services with on-call rotations, which entails being available for calls during both core and non-core business hours. If a team has an on-call rotation, all engineers on the team are expected to participate in the rotation as part of their employment. Applicants are encouraged to ask for more details of the rotations to which the applicant is applying.
Requirements- Proficient in spark, sparkSQL and hiveSQL
- Experience working with Snowflake / Databricks or similar distributed compute systems
- Excellent understanding of scheduling and workflow frameworks. Prefer prior experience with Airflow
- You've used version control systems like Git and have experience with test automation and CICD
- 4+ years of SQL experience
- 4+ years of experience with schema design and dimensional data modeling
- Experience designing, building and maintaining data processing systems
- BS degree in Computer Science or related technical field involving coding (e.g., physics or mathematics), or equivalent technical experience
- 2+ years of Python or Java, Scala development experience