Project Description:We are seeking hands-on Data Engineer consultants to build out the next-generation data warehouses for the organization. To solve the data availability and access issues of all data across the organization. Enabling a graph of connectivity between 100s of data sets. We need people that are enthusiastic about enabling internal and external clients by streamlining and facilitating easy access to their critical data that is well-defined and has established transparent levels of quality. This engineer will leverage different data platforms to achieve this while providing critical input to extend data platform capabilities. Familiarity with ETL and Cloud Platform data pipeline solutions is critical, as is REST API authoring for data access.
Responsibilities:Member of the Business Date Engineering team, working to deliver Data Ingest/Enrich Pipelines, and Access APIs using common cloud technologies.Work with consumers to understand the data requirements and deliver data contracts with well-defined SLIs to track SLA agreements.Harness modern application best practices with code quality, API test Coverage, Agile Development, DevOps, and Observability and support.Maintain programming standards and ensure the usage of the pattern/template for API Proxy.Conduct code reviews and automatic test coverageStandardize the CI/CD setup for API management tools and automated deployment.Utilize problem-solving skills to help your peers in the research and selection of tools, products, and frameworks (which is vital to support business initiatives)
Mandatory Skills Description:Min 5+ years of proven industry experience; bachelor's degree in IT or related fieldsSenior-level hands-on development expertise in Python, SQL, Spark, KafkaHands-on experience in designing and developing high-volume REST using API Protocols and Data Formats.Proven experience with Snowflake (SnowPro certification would be a plus)Proven experience working in cloud data platforms such as Azure (Databricks, ADF)Working with Azure API and DB PlatformsUnderstanding of Databases, API Frameworks, Governance Frameworks, and expertise in hosting and managing platforms like: Hadoop, Spark, Kafka, BI Tools like Tableau, Alteryx, Governance Tools like Callibra, SodaStrong understanding of Twelve-Factor App MethodologySolid understanding of API and integration design principles and pattern experience with web technologies.Design object-oriented, modularized, clean, and maintainable code and creating policies in Python.Experience with test-driven development and API testing automation.Demonstrated track record of full project lifecycle and development, as well as post-implementation support activities.
Nice-to-Have Skills:Financial experience: Public and Alternatives Asset Management / or Capital markets experienceFamiliar in NoSQL\NewSQL databasesStrong documentation capability and adherence to testing and release management standardsDesign, development, modification and testing of databases designed to support Data Warehousing and BI business teamsFamiliarity with SDLC methodologies, defect tracking (JIRA, Azure DevOps, ServiceNow, etc.)Candidate must have an analytical and logical thought process for developing project solutionsStrong interpersonal and communication skills; works well in a team environmentAbility to deliver under competing priorities and pressures.Excellent organizational skills in the areas of code structuring & partitioning, commenting, and documentation for team alignment and modifications