Staff Software Engineer, Distributed Storage

Full Time
San Francisco, CA, USA
2 months ago

Airbnb was born in 2007 when two Hosts welcomed three guests to their San Francisco home, and has since grown to over 4 million Hosts who have welcomed more than 1 billion guest arrivals in almost every country across the globe. Every day, Hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way.

The Community You Will Join:

The Storage Foundation team within Airbnb’s Storage Infra org owns Airbnb's online serving storage system. The group is responsible for designing, building and operating the software and solutions listed below. With users around the world, reliability, scalability, efficiency, high availability and platform evolution are our team’s core concerns.

  • A new source-of-truth, open-source NewSql database built on top of stateful Kubernetes that hosts all the critical Airbnb user, listing and financial data, with all the essential DB capabilities on such as backup restore, CDC, multi-tenancy. This will also serve as the unified storage backend for Airbnb Infrastructure.
  • A set of managed storage services / APIs built on top of the unified backend, e.g.:
    • A highly available, low-latency, distributed, multi-tenant K/V storage layer for Airbnb user profiling, Search, ML, pricing teams, supporting million-level read QPS, 99.9+% availability across thousands of customer tables.
    • Design and deliver a new paved path for online Graph Store that would serve as a source-of-truth Graph database, as well as a low-latency, highly available querying engine powering Airbnb’s Knowledge and Social Graph.
  • A set of control planes and provisioning managements for OpenSearch and ElasticCache for hundreds of Airbnb production clusters, at million+ IOPS and million+ indexing QPS.
  • The Difference You Will Make:

    We’re looking to add Staff Engineers who are hands-on and capable of solving broad technical challenges the team has to tackle. As a seasoned technical contributor / leader, you will bring your technical expertise, experience and thought leadership to our organization, and start in either working groups below:

  • Being part of Core DB team that builds out all the essential DB capabilities such as stateful deployment of database on Kubernetes, backup restore and disaster recovery, CDC, snapshotting, quality-of-service on a multi-tenant cluster; and then migrate existing Airbnb critical DBs to the new stack. In the longer term advance Airbnb infra’s capability towards multi-Cloud and multi-Region.
  • Being part of the Managed Storage Services team (1) to complete building V2 of our KV store on top of open-source NewSql database, and adding features like multi-tenant cost attribution, large-scale bulk loading, etc, (2) to participate in driving the roadmap, implement and design next set of fully managed storage services being a distributed caching, graph storage and/or pubsub, etc. 
  • A Typical Day: 

    • Build a fault-tolerant ingestion layer on top of open-source NewSql database to ingest TBs of data with minimal impact on online traffic.
    • Work with AWS to provide a secure client that complements Airbnb’s mesh layer and manage hundreds of ElasticCache customers cost-effectively.
    • Design and implement a seamless database migration pipeline where consistency, latency and correctness are of the utmost importance.
    • Work within Storage Infra and with Knowledge Graph and Search team to design and implement bulk loading and transactional API for writes, as well as efficient serving strategies around materialized views. 
    • Participate in regular on-call and interact with key customers to understand their requirements and pain points.
    • Coach more junior members of the team and guide them on iterating on designs and solving the right problems, provide insightful design and code review feedback

    Your Expertise:

    • 9+ years of relevant industry experience in a fast paced, high growth tech environment.
    • Knack for writing clean, readable, testable, maintainable code; experience in deep diving and then owning a complex code base.
    • Hands-on experience in building and operating distributed storage and services.
    • Ability to understand and decompose large-scale distributed systems and figure out monitoring metrics, failure scenarios and debug them in an efficient manner.
    • Strong collaboration and communication skills with a customer-first mindset, ability to abstract various customer requests into a platform vision and solution.
    • Experience in designing architecture that is long-term and evolvable.
    • Knowledge of Kubernetes, public cloud platforms (AWS, ElasticSearch, ElasticCache, Google Cloud Platform, etc) and open source preferred but not strictly required.

    Your Location:

    This position is US - Remote Eligible. The role may include occasional work at an Airbnb office or attendance at offsites, as agreed to with your manager. While the position is Remote Eligible, you must live in a state where Airbnb, Inc. has a registered entity. Click here for the up-to-date list of excluded states. This list is continuously evolving, so please check back with us if the state you live in is on the exclusion list. If your position is employed by another Airbnb entity, your recruiter will inform you what states you are eligible to work from.

    Our Commitment To Inclusion & Belonging:

    Airbnb is committed to working with the broadest talent pool possible. We believe diverse ideas foster innovation and engagement, and allow us to attract creatively-led people, and to develop the best products, services and solutions. All qualified individuals are encouraged to apply.

    We strive to also provide a disability inclusive application and interview process. If you are a candidate with a disability and require reasonable accommodation in order to submit an application, please contact us at: reasonableaccommodations@airbnb.com. Please include your full name, the role you’re applying for and the accommodation necessary to assist you with the recruiting process. 

    We ask that you only reach out to us if you are a candidate whose disability prevents you from being able to complete our online application.

    How We'll Take Care of You:

    Our job titles may span more than one career level. The actual base pay is dependent upon many factors, such as: training, transferable skills, work experience, business needs and market demands. The base pay range is subject to change and may be modified in the future. This role may also be eligible for bonus, equity, benefits, and Employee Travel Credits.  

    Pay Range$200,000—$254,000 USD