Databricks Developer

Overview:The Databricks + Spark Developer plays a crucial role in harnessing the power of data using Databricks and Spark to develop and maintain efficient data pipelines. They are responsible for implementing scalable and reliable solutions that enable data-driven decision-making within the organization.Key Responsibilities:Designing and implementing robust ETL processes using Databricks and SparkDeveloping and optimizing data pipelines for large-scale data processingCollaborating with data engineers and data scientists to support their data infrastructure needsBuilding and maintaining data warehouse solutions to support business analyticsPerforming data modeling and optimization to ensure efficient data storage and retrievalTroubleshooting and resolving performance issues with data infrastructure and pipelinesImplementing security and data governance best practices within the data platformAutomating data quality checks and ensuring data consistency and accuracyCollaborating with cross-functional teams to understand data requirements and deliver solutionsMonitoring and maintaining the health of data pipelines and infrastructureDocumenting technical design and architecture of data solutionsParticipating in code reviews and providing constructive feedback to peersStaying updated with the latest advancements in Databricks and Spark technologiesProviding technical guidance and mentorship to junior team membersRequired Qualifications:Bachelor's degree in Computer Science, Engineering, or a related fieldProven experience in developing data pipelines using Databricks and SparkProficiency in ETL processes and data warehousing conceptsStrong SQL skills with the ability to write complex queries for data manipulation and analysisAdvanced programming skills in Python for data processing and manipulationExperience in data modeling and optimizing data storage for performanceDeep understanding of big data technologies and distributed computing conceptsAbility to troubleshoot and optimize data pipeline performance for efficiency and reliabilityKnowledge of data governance, security, and compliance best practicesExcellent communication and collaboration skills to work effectively in a team environmentStrong analytical and problem-solving abilities to tackle complex data engineering challengesAbility to multitask and prioritize tasks in a fast-paced and dynamic work environmentExperience with cloud platforms such as AWS, Azure, or GCP is a plusCertifications in Databricks and Spark-related technologies are desirableExperience in Agile development methodologies and version control systems

Experience Required:Data Engineering resource with Databricks (Databricks develops a web-based platform for working with Spark) experience with Python coding background & SQL,

Key Skills

• CSS
• Python
• SQL
• OOP
• ASP.NET