Principal Data Engineer
As a Principal Data Engineer, you will help drive the direction of our Data Warehouse, and be hands on with teammates across all departments to enable access to our data and empower teams to make data driven decisions about the direction of our business. You will play a pivotal role in owning the infrastructure and code for our ETL processes within our Data Platform. You will be responsible for designing, implementing, and optimizing ingestion pipelines, enabling seamless extraction and loading of data from various sources such as customer warehouses, business systems, and events messaging buses. Movable Ink collects campaign data resulting from Billions of requests served each day. Come and help us manage and make sense of the massive amount of data we’re ingesting!
Responsibilities:- Partner with internal operations teams to identify, collect, and integrate data from various business systems, ensuring comprehensive and accurate data capture.
- Design, implement, and maintain robust ETL pipelines that feed data into the Data Warehouse, ensuring high performance, scalability, and reliability.
- Conduct thorough testing of ETL processes to validate data accuracy and integrity, and troubleshoot any issues that arise.
- Facilitate the loading of data into Data Vault 2.0 models, ensuring adherence to best practices and optimizing for performance and scalability.
- Establish and track SLAs for data processing and delivery, ensuring timely and reliable access to data for all users.
- Enforce robust Role-Based Access Control (RBAC) and Row-Level Security (RLS) policies to ensure data security and compliance with regulatory requirements.
- 8+ years of professional experience as a Data Engineer, with a focus on cloud-based Data Warehouse platforms (Redshift, Snowflake, Firebolt, BigQuery).
- Proven expertise in managing and optimizing multi-petabyte, mission-critical databases, ensuring high availability, performance, and reliability.
- Proficiency in high-level programming languages, particularly Python.
- Hands-on experience with data engineering tools and frameworks, including DBT, Apache Pulsar for distributed messaging and Apache Airflow for workflow automation and orchestration.
- Strong skills in designing and implementing scalable data architectures.
- Excellent communication skills for effective collaboration with cross-functional teams.
- Strong understanding of data security best practices within an AWS environment and compliance with regulatory requirements.
- Experience in providing technical leadership, setting best practices, and driving the adoption of new technologies and methodologies. Mentoring other engineers, fostering a collaborative team environment.
Studies have shown that women, communities of color, and historically underrepresented people are less likely to apply to jobs unless they meet every single qualification. We are committed to building a diverse and inclusive culture where all Inkers can thrive. If you’re excited about the role but don’t meet all of the abovementioned qualifications, we encourage you to apply. Our differences bring a breadth of knowledge and perspectives that makes us collectively stronger.
We welcome and employ people regardless of race, color, gender identity or expression, religion, genetic information, parental or pregnancy status, national origin, sexual orientation, age, citizenship, marital status, ethnicity, family or marital status, physical and mental ability, political affiliation, disability, Veteran status, or other protected characteristics. We are proud to be an equal opportunity employer.