Please share your updated resume if you want to apply for the position below.
Job Title Solution Architect Data & Machine Learning Platform (MLP)
Location Bangalore (Onsite)
Interview Mode - Video
Duration : 6+ Months Contract to Hire
Position Overview :
We are looking for a highly skilled Solution Architect to lead the design and implementation of a Data and Machine Learning Platform spanning edge, cloud, and on-prem components. The ideal candidate will have deep experience in Azure cloud services, data engineering, edge computing, and ML lifecycle management.
Technical Expertise :
Azure Cloud Stack & DevOps :
- Azure Databricks (including ML workspace for Feature Store and Model Store)
- Azure Data Factory (ADF) for orchestration and compute
- Azure Data Lake Storage (ADLS) implementing medallion architecture (raw, bronze, silver, gold)
- Azure Event Hub : Experience in defining topics, managing consumer groups, and integrating ETL events
- Azure Streaming Analytics : Real-time data processing for telemetry and operational data
- Azure Key Vault
- Azure App Service
- Azure Container Registry (ACR)
- Azure IoT Hub for connecting edge devices
- Azure DevOps & GitHub Actions (for CI/CD pipelines)
- GitHub self-hosted runners for ML workflow automation.
Edge and On-Prem Integration
- Strong experience in OT-IT integration and data extraction from industrial systems
Edge VM deployment using :
- Docker and Portainer for container orchestration
- RabbitMQ for messaging (read/write services from edge)
- OPC UA for interfacing with PLCs (e.g., FX Filter, NH3 Compressor)
- IDMZ deployment practices and edge-to-cloud data service integration
Machine Learning Platform (MLP) and MLOps :
- End-to-end ML lifecycle implementation : Feature Engineering, Model Training & Validation, Model Export, Versioning, and Deployment
- Hands-on with ADB ML workspace, Feature Store, Model Store
- Monitoring deployed models at 1-minute intervals
- Understanding of training vs inference, cloud vs edge deployment
- Cadence for ML models (Weekly Refresh, Monthly Retrain, Quarterly Revamp)
- Use of GitHub monorepo structure for managing model code.
Data Architecture & Integration :
- Implementation of medallion architecture in the data platform
- Integration with Unity Catalog (UC) for governance, data sharing, and cataloging
- Experience with CDC tools (e.g., Aecorsoft) for real-time SAP data ingestion
- Consumption layer design for BI, ML, and operational workloads
- Familiarity with streaming and API-based ingestion from external environments
- Template-driven ingestion and mapping using configurations
Governance and Data Modeling :
- Define and enforce data governance standards using Unity Catalog and enterprise frameworks.
- Design scalable data models to support operational analytics and ML features.
- Implement policies for access control, quality, and metadata tagging across DLZ/zones.
Key Responsibilities :
- Architect Integrated Solutions : Lead architectural design across edge, cloud, and ML across zones
- Build and Govern Data Platform : Oversee ingestion, transformation, and cataloging across Raw ? Gold layers, aligned to UC.
- Enable Scalable ML Platform : Support ML teams with infrastructure for feature storage, model ops, and deployment pipelines.
- Edge Integration and Automation : Design robust and secure OT-IT interfaces with RabbitMQ, OPC UA, and container orchestration tools.
- Monitor and Optimize Pipelines : Set up real-time monitoring for ML and ETL pipelines; optimize for performance and cost.
- Governance and Security Compliance : Ensure enterprise compliance, tagging, and secure access across all zones and services.
- Lead CI/CD Automation : Use GitHub Actions and Azure DevOps to streamline deployment of ML workflows and platform components.
(ref:hirist.tech)