Previous Job
Previous
big Data Engineer
Ref No.: 18-02902
Location: Oakland, California
Position Type:Direct Placement
Start Date: 04/16/2018
 Required Skills/Qualifications:
 
3 to 5 years of relevant experience
  • Bachelor's and/or Master's degree in computer science or equivalent experience
  • Strong communication, analytical and problem-solving skills with a high attention to detail
 
Desired Experience:
 
  • Thorough understanding of the entire Hadoop Ecosystem
  • Should have created convoluted data-pipelines
  • Should have expertise with more than one of the following Hive, Map Reduce, Pig, Oozy, Spark
  • Should have worked with clusters from different distributors namely MapR, CloudEra, Hortonworks
  • Should have worked extensively on data modeling and data lake creation
  • Should be able to estimate, design pipelines based on High data volumes and create base data assets for analytical work to commence
  • Proficient with Ubuntu/Linux and shell scripting
  • Should be aware of one Scripting language namely Python/Scala/Java
  • Expert in performance tuning Hive queries based on storage formats and partitioning
  • Strong fundamentals in SQL
  • Experience with Streaming data sources and NoSql databases is a plus
 
 
Responsibilities:
 
  • Develops data architectures and pipelines by;
  • Understanding requirements;
  • Understanding deadlines;
  • Understanding systems flow, data usage, and work processes;
  • Working in tandem with the Analytics team and providing inputs as and when required
  • Investigating problem areas;
  • Follow Agile and Scrum practices;
  • Be thought partner to the clients in creating the data charter
  • Should be able to lead client discussions and answer design specific queries
  • Documents and demonstrates solutions by developing documentation in the forms of flowcharts, layouts, diagrams, charts, code comments and clear code;
  • Provides information by collecting, analyzing, and summarizing development and service issues;
  • Accomplishes engineering and organization mission by completing related results as needed and on time with high quality;