Lead Big Data Developer

Job descriptionTitle: Lead Big Data DeveloperNumber of positions: 1Duration: Long Term (1+ Year)Location: RemoteTerms: The client requires U.S. citizens or Green Card holders to work directly on a 1099 basis. (No H1B or OPT candidates or 3rd party companies).Please apply if you have relevant real-time experience. Background and employment verification will be conducted.Interview Process: There will be three rounds of technical interviews via video call, including live coding scenarios.If you are interested, please send your resume to hr@birsolutions.com. Requirements:6+ years of development experience in Java AND Scala working on ETL projects in an Agile environment4+ years of experience in big data projects3+ years of experience with API development (REST API’s) and processing with JSON and csv files2+ Years of experience working in a Lead role supporting design of applications and ETL projectsStrong knowledge of big data technologies including: AWS EMR, AWS EKS, Apache Spark, SQL & serverless technologies like AWS Dynamo DB, AWS LambdaUnit testing using JUnit or ScalaTestDesired:Business Process Management experience using AWS functionsPython & Shell ScriptingExperience with performance testing tools such as Gatling or JmeterResponsibilities:Comprehend Complex Business RequirementsDesign and develop ETL pipelines to collect, validate, and transform data according to the requirementDevelop automated unit tests, functional tests and performance tests.Support optimal data pipeline architectureDesign ETL jobs for seamless execution in AWS cloud environmentReduce processing time and cost of ETL workloadsLead cross-functional review meetings with peers and stakeholdersContinuous data quality checks.Identify areas where machine learning can be used to identify data anomalies

Key Skills

• AWS
• Java
• SQL
• REST APIs
• Agile