IMPACT 2022 – Managing 50 Billion Things
October 28, 2021
IMPACT 2022 – Balancing Kubernetes performance, resilience & cost by using ML-based optimization – a real-world case
November 1, 2021
Data Engineer, Infrastructure Strategy
How would Facebook scale to the next billion users? The Infrastructure Strategy group is responsible for the strategic analysis to support and enable the continued growth critical to Facebook’s infrastructure organization.
We are looking for a Data Engineer to not only build data pipelines but also extend the next generation of our data tools. As a Data Engineer, you will develop a clear sense of connection with our organization and leadership – as Data Engineering is the eyes through which they see the product.
This is a partnership-heavy role. As a member of Infrastructure Strategy Data Engineering, you will belong to a centralized Data Science/Data Engineering team who partners closely with teams in Facebook’s Infrastructure organization. Through the consulting-nature of our team, you will contribute to a variety of projects and technologies, depending on partner needs. Projects include analytics, ML modeling, tooling, services, and more.
- Partner with leadership, engineers, program managers and data scientists to understand data needs.
-
Design, build and launch extremely efficient and reliable data pipelines to move data across a number of platforms including Data Warehouse, online caches and real-time systems.
-
Communicate, at scale, through multiple mediums: Presentations, dashboards, company-wide datasets, bots and more.
-
Educate your partners: Use your data and analytics experience to ‘see what’s missing’, identifying and addressing gaps in their existing logging and processes.
-
Broad range of partners equates to a broad range of projects and deliverables: ML Models, datasets, measurements, services, tools and process.Leverage data and business principles to solve large scale web, mobile and data infrastructure problems.
-
Build data expertise and own data quality for your areas.
-
5+ years of Python development experience.
-
5+ years of SQL experience.
-
3+ years of experience with workflow management engines (i.e. Airflow, Luigi, Prefect, Dagster, digdag.io, Google Cloud Composer, AWS Step Functions, Azure Data Factory, UC4, Control-M).
-
3+ years experience with Data Modeling.
-
Experience analyzing data to discover opportunities and address gaps.
-
5+ years experience in custom ETL design, implementation and maintenance.
-
Experience working with cloud or on-prem Big Data/MPP analytics platform(i.e. Netezza, Teradata, AWS Redshift, Google BigQuery, Azure Data Warehouse, or similar).
-
Experience with more than one coding language.
-
Designing and implementing real-time pipelines.
-
Experience with data quality and validation.
-
Experience with SQL performance tuning and e2e process optimization.
-
Experience with anomaly/outlier detection.
-
Experience with notebook-based Data Science workflow.
-
Experience with Airflow.
-
Experience querying massive datasets using Spark, Presto, Hive, Impala, etc.
