Spark Scala & Java Developer

Spark Scala & Java DeveloperShift- Malaysian / EURExp- 5+ yrs (2 Positions), 9- 11 yrs (2 Positions)

Full time Contract OpportunityFully Remote

please share resume at Khalil@weradioactive.com

Responsibilities-Create Scala/Spark jobs for data transformation and aggregationProduce unit tests for Spark transformations and helper methodsWrite Scaladoc-style documentation with all codeDesign data processing pipelines

Skills-Scala (with a focus on the functional programming paradigm)Scalatest, JUnit, Mockito, Embedded CassandraApache Spark 2.xApache Spark RDD APIApache Spark SQL Data Frame APIApache Spark MLlib APIApache Spark GraphX APIApache Spark Streaming APISpark query tuning and performance optimizationData Modelling, Data Warehousing, Hadoop, Hive, ETL, JavaSQL database integration ,Microsoft, Oracle, Postgres, and/or MySQLExperience working with HDFS, S3, Cassandra, and/or DynamoDBDeep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)

Key Skills

• Java
• SQL