Spark & Scala Developer

Spark & Scala DeveloperExp- 5-8 yrsShift- MalaysianRemote - One PositionOnsite (Malaysia) - One Position

Full time Contract OpportunityFully Remote

please share resume at Khalil@weradioactive.com

Responsibilities-Create Scala/Spark jobs for data transformation and aggregationProduce unit tests for Spark transformations and helper methodsWrite Scaladoc-style documentation with all codeDesign data processing pipelines

Skills-Scala (with a focus on the functional programming paradigm)Scalatest, JUnit, Mockito {{ , Embedded Cassandra }}Apache Spark 2.x{{ Apache Spark RDD API }}{{ Apache Spark SQL DataFrame API }}{{ Apache Spark MLlib API }}{{ Apache Spark GraphX API }}{{ Apache Spark Streaming API }}Spark query tuning and performance optimizationData Modelling, Data WarehousingSQL database integration {{ Microsoft, Oracle, Postgres, and/or MySQL }}Experience working with {{ HDFS, S3, Cassandra, and/or DynamoDB }}Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)

Key Skills

• SQL