Source description
About the role
Architect, Design, develop, and maintain data ingestion, transformation, and orchestration pipelines (batch and real-time).
Build and optimize data Lakehouse architectures using Azure Synapse, Delta Lake, or similar frameworks.
Integrate and manage structured and unstructured data sources (SQL/NoSQL, files, documents, IoT streams).
Develop and operationalize ETL/ELT pipelines using Azure Data Factory, Databricks, or Apache Spark.
Collaborate with Data Scientists to prepare and serve ML-ready datasets for model training and inference.
Implement data quality, lineage, and governance frameworks across pipelines and storage layers.
Work with BI tools (Power BI, Superset, Tableau) to enable self-service analytics for business teams.
Deploy and maintain data APIs and ML models in production using Azure ML, Kubernetes, and CI/CD pipelines.
Ensure scalability, performance, and observability of data workflows through effective monitoring and automation.
Collaborate cross-functionally with engineering, product, and business teams to translate insights into action.
Mentor junior team members and review their work
More at ValGenesis