Praful Jain

Architected enterprise-scale Databricks Data & AI platforms using Unity Catalog, Delta Live Tables (DLT), and Serverless Compute.

Led GenAI and Agentic AI initiatives, developing RAG-based solutions and AI agents for natural language interaction with enterprise data.

Delivered 25% storage savings, 30% compute cost reduction, and 36% pipeline cost optimization through governance and platform modernisation.

• Improved workload performance by 53% using Photon, Adaptive Query Execution (AQE), and Databricks optimization best practices.
• Built AI-powered cost observability dashboards using Databricks SQL and Genie, enabling proactive FinOps governance.
• Created Databricks Apps, data products, GenAI capabilities, and AI agent workflows to enhance data utilisation.
• Implemented CI/CD pipelines using Azure DevOps and Databricks Asset Bundles (DAB), enabling automated deployments and environment promotion.
• Led and mentored a team of 8 engineers, driving technical delivery, conducting architecture reviews, and enhancing stakeholder engagement.
• Designed secure Snowflake-ADLS integrations and automated near real-time ingestion using Snowpipe and event-driven architectures.
• Designed and implemented agentic application for data table health solution on Databricks, improving data integrity and access.
• Defined key responsibilities and deliverables in role.
• Client: McKesson (PSaS)
• Projects: Genesys, Environmental Social & Governance(ESG), DHGA, OneAI & FBL(Feed Back Loop)

Established Snowflake storage integration to securely ingest landing data from ADLS.

Implemented Snowflake notification integration to auto-trigger Snowpipe on new file arrivals.

Configured Snowpipe for continuous loading of raw data into intake tables.

• Developed and maintained stored procedures for data transformation, cleansing, and upserts into target tables.
• Improved execution time by 30% in Phase 2 delivery through optimization efforts.
• Led projects to analyse Tranvoyant data, driving insights for decision-making
• Acted as primary contact for McKesson client, facilitating effective communication and project alignment
• Defined key responsibilities for role to ensure clarity and accountability within the team

Built end-to-end data pipelines to load and process data into Elasticsearch for robust data management.

Optimised 60+ Spark jobs, achieving 33% reduction in processing time.

Extracted data from Snowflake, Azure SQL, Oracle, and Blob using SnowSQL and Sqoop into Hive/Spark warehouse.

• Implemented Spark jobs to fetch data from SOAP APIs and store in Hive.
• Created custom Elasticsearch mappings using Python.
• Developed Power BI dashboards to effectively visualise data from Blob storage, enhancing data accessibility.
• Provisioned transient GCP Dataproc clusters for on-demand processing.
• Automated Dataproc cluster lifecycle using Google Cloud Functions.
• Projects: McKesson Business Analytics(MBA), Gaps For Growth (G4G)
• Client: McKesson
• Key Responsibilities:

• Developed web-based project using Spring and Hibernate frameworks, delivering 86 reports for client, including daily employee activity, and integrating with ERP system.
• Designed RESTful APIs for Java applications, enabling seamless integration with front-end technologies and third-party services.
• Applied Java design principles in designing scalable, robust and reliable components to minimise business disruptions.
• Created detailed documentation for Java applications, simplifying maintenance and future enhancements.