Sr. Spark Developer

Job Title: Sr. Spark DeveloperLocation: Remote (Halifax Nova Scotia, Canada)Description: We are seeking a highly skilled Sr. Spark Developer with a strong background in application development. The ideal candidate will have extensive experience with Spark, Python, and Databricks, and possess the ability to work in an Agile-driven environment. This role requires excellent problem-solving skills and the ability to communicate technical concepts clearly.

Responsibilities:Develop and maintain large-scale data solutions using Spark and Python.Utilize Databricks for data processing and analytics.Work with Azure AI Studio to implement AI models and solutions.Debug issues using stack traces, log files, and other system outputs.Proactively provide technical solutions to enhance system performance and functionality.Stay updated with new technologies and apply them to deliver added business value.Collaborate with cross-functional teams in an Agile environment.

Required Skills and Experience:5+ years of hands-on application development experience.5+ years of experience with Spark and Python.5+ years of experience with Databricks (DBRX models).3+ years of experience working with Azure AI Studio.Strong debugging skills.Proficiency in Agile development methodologies.

Preferred Experience:Experience with LLM inferencing using Azure OpenAI, including OpenAI setup (APIM and Cosmos DB RUs).Experience with Quart-based Python applications.Experience with Cosmos DB for logging.

Key Skills

• Python
• Azure