Previous Job
Data Engineer
Ref No.: 18-08108
Location: Foster City, California
Job title: Data Engineer
Location: Foster City, CA
Duration: 6+ months

Looking for a skilled Data Engineer who is well-experienced with SQL/HiveQL/Python/Shell Scripting. This position is going to support the BI Development team within Client with direct exposure to business users.

We are looking for someone with solid background in creating and organizing data sets and has worked with large datasets in creating custom logic, flags and summary tables, as well as automation of workflow using Oozie.

Skills: Skilled in more than one of SQL/HiveQL/Shell Scripting/Python

o Bachelors degree in computer science, software engineering or relevant field required
o 5-7 years total experience required

Responsibility level
o Exercises independent judgment with minimal direction from supervisor


o 4+ years' experience with pulling together data sets from multiple sources (Required)
o SQL/HiveQL/Shell Scripting, Data transformations, creating custom aggregations from large datasets (Required)
o Experience incorporating multiple data sources into the Tableau data model in support of the reporting (Nice to have)
o Verbal and written communication skills, problem solving skills, customer service and interpersonal skills
o Knowledge of BI and data warehousing principles including data modeling, data quality, extract/transform/load process, job scheduling and indexing (Required)
o Strong ability to work independently and manage one's time
o Ability to execute fast-paced and dynamic projects in a highly organized manner on a tight schedule
o Strong ability to troubleshoot issues and make system changes as needed to resolve issue

Major Job Duties and Responsibilities:
o Support custom BI development for our internal and external clients
o Develop datasets from existing data sources in Hive by creating custom aggregations and flags that will be fed into Tableau dashboards
o Connect the dashboard directly to data sources on Hadoop and refresh them on an ongoing basis by using schedulers