Senior Big Data Engineer
Previous Job
Previous
Senior Big Data Engineer
Ref No.: 17-09356
Location: Cupertino, California
Position Type:Contract
Start Date: 10/03/2017
Title: Senior Big Data Engineer
Type: Contract to Hire
Location: Santa Clara, CA


Job Summary
As a member of our client's team, you will be faced with highly complex issues in a large scale, distributed system environment. In order to ensure a reliable and rewarding experience, you will be empowered to develop and design new solutions heavily focused on system automation. We look for talented engineers in both the Operations and Development space to bring these unique solutions to production at a rapid pace. From an Operations perspective, this makes us the most successful personal assistant in the industry.

Key Qualifications
  • Knowledge of the Linux operation system (OS, networking, process level)
  • Understanding of Big Data technologies (Hadoop, Hbase, Spark, Kafka, Flume, Hive, etc)
  • Understanding of one or more object-oriented programming languages (Java, C++)
  • Fluent in at least one scripting language (Shell, Python, Ruby, etc.)
  • Strong verbal and written communication skills
  • Passionate about being a part of a tight-knit Operations team
Description
The Data Engineering Operations team manages the platform for both ingestion and analytics of worldwide events. To accomplish this, we build automation tools and services to prevent failures and page out individuals when there really is a problem, not just noise. Our engineers not only work closely with Operations, but also with the development and analytics engineers, as well as outside organizations. We build data pipelines for maximum efficiency, scalability and reliability to allow domain specific engineers to focus on their specialties. A successful candidate will have experience in being a Systems Administrator that has moved on to development and automation in their career. - Operate our client's largest infrastructure supporting millions of customers at double digit PB scale - Troubleshoot complex issues across the entire stack - Design and develop automation frameworks to handle both development and production events at scale - Advise other teams on technical direction.

Education
B.S. degree in Computer Science or 3+ years of building data pipelines experience or equivalent.

Additional Requirements
(Nice to have) - Java JVM performance tuning/optimizations - Mesos compute platforms and job schedulers