Previous Job
Spark/Scala Developer
Ref No.: 18-13991
Location: Durham, North Carolina
Position Type:Contract
Scala and Spark Developer
Durham, NC
Through end of Dec to start

- Developer with hands-on experience in Spark and Scala programming language.
- Experience in implementing batch and real-time Big Data integration frameworks and/or applications, in private or public cloud, preferably AWS, using Spark, Hadoop
- Experience and knowledge with a strong background in the developing, tuning and performance of complex Spark workflows
- Knowledge of Spark streaming
- Experience building data pipelines in Spark
- Should have hands-on experience on Informatica Cloud and Real Time
- Good knowledge of data-warehousing/ETL concepts & data-modeling concepts (Entity Relationship, Multi-dimensional) is required.
- Good understanding of dimensional modeling and ETL designs for loading Star Schemas
- Experience in writing complex SQL to implement business rules, extract and analyze data
- Produce data building blocks, data models, and data flows for varying client demands such as dimensional data, data feeds, dashboard reporting, and data science research & exploration. Big data addition.
- Define and build data cleansing and imputation, map to a common data model, transform to satisfy business rules and statistical computations, and validate data content.
- Strong experience in Information Systems, including software design/development in: Informatica, PL/SQL, UNIX, Scripting, Oracle, and MSSQL Databases.
- Experience integrating Informatica with other systems
- Build end to end data flows from sources to fully curated and enhanced data sets. This can include the effort to locate and analyze source data, create data flows to extract, profile, and store ingested data