Our Opportunity:

We are looking for a highly qualified and forward-thinking Lead Data Engineer to join our Enterprise Data Systems team, which drives reporting, analytics, and data science capabilities for numerous teams across the organization. In this key role, you will be responsible for crafting and building a groundbreaking data platform that will serve as the foundation for data processing and analysis! The ideal candidate will bring extensive experience in developing large-scale data ingestion and processing frameworks, coupled with a strong command of data engineering standard methodologies, while also providing strategic leadership and mentorship to the team.

Key Responsibilities:

• Lead the design, development, and optimization of scalable, efficient, and reusable data ingestion and processing frameworks using Spark and Spark Streaming.
• Architect, implement, and run data pipelines using a variety of tools, including Kafka, Kinesis, Airflow, and AWS services such as Glue, EMR, S3, SQS, SNS, and Step Functions.
• Guide and mentor the data engineering team, setting standard processes, technical guidelines, and guardrails to ensure high-quality data solutions.
• Collaborate with multi-functional teams, including data science and analytics, to craft and build data platforms that support the organization’s evolving data needs.
• Ensure data quality, observability, and governance by implementing best-in-class practices for data auditing, validation, and monitoring.
• Hands-on experience working with data warehouse platforms such as Snowflake, Redshift, and relational databases like DynamoDB and PostgreSQL.
• Develop and lead orchestration pipelines using Airflow to optimize job scheduling and improve operational efficiency.
• Drive innovation by evaluating new data tools, technologies, and frameworks to enhance the company’s data infrastructure.
• Implement and maintain CI/CD pipelines and infrastructure-as-code using Terraform for fast, efficient deployments.
• Collaborate on cloud architecture decisions and drive the use of AWS services to lead large-scale data processing.
• Partner with collaborators across the organization to understand their data needs and deliver reliable, high-quality data products.

Qualifications:

• Shown experience as a Lead Data Engineer or Senior Data Engineer in a data-driven environment.
• Highly skilled and hands-on team member to design, and develop data pipelines using Python, Spark, Spark streaming and Kafka for building large-scale data processing frameworks.
• Proficiency in cloud data platforms like AWS Glue, EMR, S3, SQS, SNS, and Step Functions.
• Proven understanding of Snowflake, Redshift, DynamoDB, PostgreSQL, and other data warehouses/databases.
• Experience with Airflow for orchestration and automation of data workflows.
• Deep understanding of data quality, governance, and observability principles.
• Knowledge of CI/CD pipelines and terraform for cloud infrastructure management.
• Proven understanding of data platforms such as AWS Data platform, Databricks, Cloudera.
• Ability to work in a fast-paced, multi-layered environment, with excellent problem-solving and leadership skills.

Nice to Have:

• Familiarity with Databricks or Cloudera platforms.
• Experience setting up and deploying data infrastructure using Terraform.
• Previous experience working with CI/CD pipelines to streamline deployment processes.

Why Join Us: You will be a key player in building a data platform that supports a data-driven organization, impacting reporting, analytics, and machine learning. You’ll work with innovative tools and lead a high-performing team, helping shape the future of data infrastructure within the company.

Chewy is committed to equal opportunity. We value and embrace diversity and inclusion of all Team Members. If you have a disability under the Americans with Disabilities Act or similar law, and you need an accommodation during the application process or to perform these job requirements, or if you need a religious accommodation, please contact CAAR@chewy.com.

If you have a question regarding your application, please contact HR@chewy.com.

To access Chewy's Customer Privacy Policy, please click here. To access Chewy's California CPRA Job Applicant Privacy Policy, please click here.