From Hype to Impact: How One AI Expert Transformed Work For 2,500 Employees

Artificial Intelligence is on everyone’s lips today, but when it comes to real-world success, the stories are far fewer than the hype suggests.

The truth is that while building experimental AI systems is relatively easy, bringing them into production and ensuring they serve thousands of users reliably is an entirely different challenge. What does it actually take to roll out an enterprise-grade chatbot that works; not just once in a demo, but day after day for an entire organisation?

Vignesh is an applied AI engineer at a leading AI startup and an independent researcher who has made a career out of taking ambitious ideas and turning them into production-ready AI systems.

His talk on evaluating and improving the performance of agentic systems attracted a massive crowd of eager practitioners looking for ways to get their GenAI prototypes into the hands of end users.

A graduate of Stanford’s AI Professional Programme, Vignesh has become a familiar face at conferences and hackathons in London, where his work has repeatedly stood out for its innovation and reliability.

 

Building Ask Polly

 

One of Vignesh’s most successful projects, an enterprise-grade AI assistant he built called Ask Polly, which successfully reached the hands of more than 2,500 employees across a global organisation.

Ask Polly quickly became a trusted daily companion that unblocked teams and delivered measurable efficiencies, conservatively estimated at over $2 million in time and productivity savings.

 

Ask Polly: An Engineering Marvel

 

At its heart, the enterprise AI assistant Vignesh built is designed to make knowledge effortlessly accessible. Employees can simply type a question; whether about their products, processes, or shared resources and receive instant, reliable answers.

The assistant has access to all internal knowledge bases, documentation, marketing content, and enablement material. Beyond just text responses, Ask Polly can surface relevant presentations and documents, saving teams hours of digging through folders. It also helps teams find images and slides from thousands of internal PowerPoint decks. Integrated directly into Slack, it fits naturally into daily workflows and has quickly become a trusted companion.

What makes Ask Polly stand out is its distinctive engineering architecture. From day one, the AI assistant was designed to operate entirely self-contained, without reliance on frontier models like ChatGPT or Claude. Instead, it runs in a fully self-hosted environment, a necessity driven by the organisation’s strict data protection requirements.

For businesses handling sensitive information, building AI systems without state-of-the-art language models powering them is notoriously difficult. Ask Polly overcomes this challenge while running on a remarkably modest infrastructure: just a single GPU. This efficiency is made possible by an innovative inference approach developed by Vignesh, combining model quantisation with continuous input batching.

Tackling The Retrieval Challenge

 

The engineering pipeline for a typical RAG (Retrieval-Augmented Generation)-based AI assistant involves two distinct phases.

The first phase, called retrieve (or search), is when the assistant looks through the internal documentation for reference material. The second phase, called generation (or response), is when the AI model uses the retrieved information to compose a response to the user.

A critical challenge with most enterprise AI assistant applications across the industry today is the low accuracy of retrieval/search results.

Vignesh realised this early on and applied a number of tuning techniques to continuously improve Ask Polly’s retrieval accuracy, including fine-tuning the underlying search (embedding) model to the unique needs of the business, deterministic metadata-based filtering, and routing layers that understand user intent and route queries through distinct paths.

 

150X ROI In Less Than 1 Year

 

The result is not only technical elegance but also dramatic cost savings. An off-the-shelf solution from a vendor in this space would have cost the company millions per year. Vignesh reports that maintaining Ask Polly costs less than $1,000 per month while delivering a return on investment exceeding 150x for the organisation.

 

Start With The End User In Mind

 

Vignesh says he began not with the technology but with the user. Every decision stemmed from a simple but powerful set of questions: What exactly does the end user want? Where does that information live?

How should the interaction feel natural and intuitive? By blending his deep technical expertise with empathy for the user journey, Vignesh built a solution that didn’t just answer questions but genuinely fit into people’s workflows.

Just as importantly, he championed evaluation from the very first stages of development. Speaking to over 200 attendees at the Gartner Data & Analytics Summit earlier this month, he emphasised that testing an AI system isn’t only about accuracy, it’s about tone, clarity, consistency, hallucination risks and the overall impact on the end user.

This holistic lens on evaluation was refreshing in a field often obsessed with benchmarks alone, and it proved to be one of the keys to delivering a chatbot that employees trusted and relied upon every single day.

 

Recognition and Business Impact

 

The impact of the rollout was undeniable. With over 2,500 employees actively using the chatbot, it quickly became the go-to companion for answering questions that once required digging through dense documents and endless folders. Thousands of queries were resolved seamlessly, translating into hundreds of hours of productivity gained each month.

Remarkably, this was achieved at an annual maintenance cost of under $15,000, while the estimated value unlocked for the business reached close to $2 million, a return on investment that speaks for itself. The recognition was just as powerful as the results. Vignesh reports that the chatbot was celebrated as the most innovative project within the organisation, praised for transforming how teams accessed knowledge.

From the CEO to the newest field sales recruit, employees at every level relied on it daily—a rare sign that an AI solution had truly crossed the threshold from novelty to necessity. Beyond building the system, Vignesh says he worked closely with cross-functional teams to ensure they understood how to use it effectively, guiding them through both the possibilities and the limitations.

 

The Human Side of AI

 

For him, the real magic lies in how AI can serve as a force multiplier—leveling the playing field and giving every employee, regardless of role or background, the ability to access knowledge instantly. He has watched people who once struggled with information overload suddenly excel, empowered by a tool that amplifies their capabilities.

That, he says, is the deepest satisfaction: not just building AI that works, but bringing AI into production in a way that helps people work and grow better every day.