| Role & Responsibilities fro Data Engg. JD for Grade 9 |
| · Diverse Data Engineering activities on Azure Data platform. |
| · Build & Deliver Data pipeline connecting various enterprise data sources both RDBMS, NoSQL & APIs. Develop data mappings to existing data sources. |
| · Design and develop big data processing notebooks using Azure Databricks |
| · Scripting and programming using programming languages such as Python, PySpark etc. |
| · Lead the data identification, data analysis efforts for data sources and work directly with data owner teams |
| · Lead a team of junior Data Engineers in an Agile/Scrum setting |
| · Develop data requirements for new data sources. |
| · Design and development of data extraction, data ingestion, data quality rules implementation |
| · Understand the data model, transform data to target schema from relational and semi structured source data. |
| · Clean and process the data for Machine Learning consumption. |
| · Provide Business Intelligence (PowerBI) and Data Warehousing (DW) solutions and support by leveraging project standards and leading analytics platforms |
| · Evaluate and define functional requirements for BI and DW solutions |
| · Design and development of data extraction, data ingestion, data quality rules implementation |
| · Involvement in architecture and design of data |
| · Work with Data Governance team to implement Data Quality and Security guidelines |
| · Work with DevOps team to implement CI/CD pipelines |
| |
|
|
| Required Skills :- |
| |
|
|
| 8+ years overall experience preferably in data domain (data analysis, database developer) |
| Minimum 5 years’ experience as a cloud-based Data Engineer |
| Minimum 3 years’ experience and strong knowledge of Azure Data Services such as Azure Databricks, Delta Lake, Azure Data Lake Storage, Blob Storage, Azure Data Factory |
| Minimum 2 years’ experience on Analytics Visualization tools like Power BI, Qlik Sense etc. |
| Strong knowledge of Big Data and Analytics, including Apache Spark, Delta Lake |
| Good understanding of data integration and warehousing (ETL, ELT) processes |
| Strong knowledge of Database concepts, SQL and experience in working with Database clients like, TOAD, SQL developer, SQL Server Management Studio |
| Good knowledge of NoSQL/ Document DB. |
| Good understanding of Enterprise Information Model and Enterprise Data Warehouse |
| Proficient in analyzing business requirements and mapping them to technical requirements |
| Data Analysis and Interpretation, and Data Issue Debugging Skills |
| Good knowledge of DevOps tools and processes as it applies to Data Engineering |
