Senior Data Engineer - 13623
Woodland Hills, United States
Alpharetta, United States
Chicago, United States
Denver, United States
New York, United States
Pleasanton, United States
As a member of the Data Operations team, you will help advance the platform to allow the organization to make data-driven decisions and improve the product experience. To achieve this, the team currently leverages the power of the Google Cloud Platform and existing open-source technologies.
The team is also responsible for building and operating tools and platforms around data while also promoting best practices around it with a focus on user data privacy. We are looking for a driven, detail-oriented, and passionate engineer to come to join our new Data Operations team.
Roles and Responsibility (list in order of importance)
The Sr. Data Engineer will;
- Ensure 99.99%+ availability of the services and infrastructure that spans across multiple global datacenters in private and public clouds.
- Design, build and maintain production data-pipelines (both batch and streaming) that deliver over a billion events with measurable quality within bounds of defined SLA(s)
- Work with product and engineering managers to build necessary tools to help with their data-related needs
- Build tools and infrastructure that enables external teams (engineers and analysts) to effectively create efficient data-pipelines
- Be a champion of the overall strategy for data governance, security, privacy, quality, and retention that will satisfy business policies and requirements
- Identify, document, and promote best practices
- Monitor and maintain health, performance, and security of all infrastructure components.
- Build systems and perform necessary tasks to deliver against committed project timelines.
- Desire to automate everything
- Solve real-life problems in a bleeding-edge, high-performance, and high-traffic environment.
- Maintain documentation and operational knowledge base.
- Adhere to the change management and other established processes and procedures.
- Respond to and troubleshoot incidents. Conduct root cause analyses.
- Evaluate and analyze systems, performance, issues, and metrics to provide recommendations for continuous improvements.
- Participate in a scheduled 24/7 on-call rotation for second tier support escalations.
Technical/Specialized Knowledge, Skills, and Abilities:
- 4+ Years of Data engineering experience
- 4+ years of hands-on experience in working with data, including but not limited to Data Warehouse (DWH) environment with data integration / ETL of large and complex data sets. Building large scale data-processing systems with experience in
- 2+ years supporting a Saas/Hosting type, critical revenue-generating environment.
- 2+ years interacting with APIs (REST, SOAP, etc.)
- Big Data technologies such as MapReduce, Hadoop, Spark, Kafka, or GCP/AWS equivalents
- Programming experience in SQL and Python or Java
- Hands-on experience in data modeling, data pipeline design and development
- Proficient in relational as well as non-relational data stores, methods and approaches.
- Hands-on problem-solving skills, technical leadership and mentoring qualities
- Familiarity with scripting and or systems programing languages (e.g. Bash, PowerShell, Chef, Ansible, etc.)
- Strong written and oral communication skills
- Ability to participate in On-Call rotation
- A minimum of two years of experience in a 24x7 operations organization, deploying and operating complex data infrastructure at scale
- Experience with scripting and/or systems programming languages (eg: Python, Bash, PowerShell, Chef, etc)
- Experience with JSON
- Significant experience with open-source platforms and technologies
- Experience with Agile software development processes and methodologies
- Experience with Jira
- Experience with Google Cloud or similar cloud provider
- Track record of architecting, developing, implementing robust, distributed online solutions
- Experience in mixed Windows/Unix/Linux environments
Work is primarily sedentary in nature; no special demands are required.