Big Data Developer

Data Engineer Big Data

Job Description Develops and maintains scalable data pipelines Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization. Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it. Defines company data assets (data models), spark, sparkSQL, and hiveSQL jobs to populate data models. Designs data integrations and data quality framework. Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.Qualification: Bachelor's Degree in Computer Science or related field 5+ years of work experience Strong experience in SQL ( include complex SQL query , SQL performance tuning , Index , Lock ) Experience with schema design and dimensional data modeling Experience with Hive SQL , Spark(Spark SQL, DataFrame) Experience with near-realtime data warehouse (10-30 mins level) Experience with data quality check Experience in Java or Python or Shell Script

Key Skills

• SQL