Building a RAG-Powered AI ChatBot from Scratch

Most AI chatbots today are all talk.

They’re marketed as intelligent assistants, but when you actually ask them something specific - like “What are your return policies?” or “Do you offer a discount for teachers?” - they either guess, hallucinate, or send you in circles.

That’s because nearly all these bots rely solely on large language models like GPT-3.5 or GPT-4 with zero knowledge of your business. They’re trained on the internet - not on your documents, your policies, or your unique customer workflows.

So what happens?

• Businesses install them on their site, expecting magic.

• Visitors ask questions the bot can’t answer.

• Conversations go nowhere. Leads drop off. Frustration builds.

• Eventually, the chatbot is disabled or ignored - and the business is back to square one.

This is a familiar story - and one I wanted to fix.

For Week 19 of Project52, I set out to build something better:
A Retrieval-Augmented Generation (RAG)-based chatbot that actually understands the business it represents.

Not a dumb bot.
Not a hallucination engine.
But a self-hosted, private chatbot trained on each business's documents - and ready to answer user questions with actual context.

In this project, I built:

• A secure multi-tenant SaaS system

• Admin dashboard for lead capture and file uploads

• Per-admin chatbot interface with document-trained intelligence

• A complete backend pipeline for RAG using LangChain and ChromaDB

• A GPT fallback system that gracefully handles unrelated or out-of-context queries

What started as a simple idea - “Let’s let users train GPT on their PDFs” - evolved into a full-fledged product that solves a real business pain: turning conversations into conversions with intelligence, not guesswork.

And the entire thing was built from scratch - frontend, backend, database, authentication, chatbot logic, and document processing - in one intense week.

This is what I’ll break down in this edition.

Watch it in action:

What is RAG - and Why Does It Matter?

Large Language Models (LLMs) like GPT-4 are incredibly powerful. They can write code, answer questions, summarize books - but here’s the catch:

If you upload your PDF handbook, pricing sheet, refund policy, or onboarding guide - they don’t magically absorb that information. Out of the box, GPT can’t retrieve or reason based on your documents.

That’s where RAG - Retrieval-Augmented Generation - changes the game.

The Problem RAG Solves

Let’s say you’re a customer asking:

A regular GPT-based chatbot might respond with a generic, helpful-sounding (but false) answer like:

This is a hallucination. Your policy might say otherwise.

Now imagine that same question asked to a chatbot powered by RAG.

Here’s what happens instead:

1. Your actual refund policy (from a PDF you uploaded) is chunked, embedded, and stored in a vector database like ChromaDB.

2. When the question is asked, the chatbot retrieves the most relevant chunks from your document.

3. GPT then uses these real excerpts to ground its answer in your specific policy.

So now it replies:

The answer is now truthful, contextual, and traceable.

Why RAG is the Foundation of Future Chatbots

RAG is the architecture behind the most trusted AI systems used in enterprise today. It's how:

• Support bots answer ticket questions without misleading users

• Internal tools surface knowledge from private documentation

• Chat interfaces stay grounded in your data, not the internet’s guesses

In this project, RAG is the core engine - enabling each admin’s chatbot to only answer based on documents they’ve uploaded.

This ensures:

• No hallucinations

• No privacy violations

• No wrong answers pulled from unrelated contexts

Just hyper-relevant, document-backed responses that build trust and convert better.

Phase 1 - User Authentication and Admin Isolation

Before we could even think about building an intelligent chatbot, we needed to lay the foundation for secure access and private data control. In a multi-tenant system-where different businesses each use the same platform-data isolation is absolutely critical. You can’t have one business accessing another’s leads or documents.

This phase was all about creating independent admin environments while maintaining a shared codebase and infrastructure.

Key Features Built

1. Admin Sign Up and Login System
We implemented secure registration and login using hashed passwords, token-based authentication (JWT), and input validation. Each admin creates their own account, and upon login, receives a secure token used to access protected endpoints.

2. Isolation by Admin Email
The entire backend is designed to tie every piece of data-leads, documents, chat history-to the admin’s unique email. That means when an admin logs in, they only see data that belongs to their account. No cross-contamination.

3. Bot Protection with Math Challenge
To protect against bot signups, we added a simple math challenge at registration. This low-friction method weeds out automated spam bots without requiring third-party CAPTCHAs.

4. Rate Limiting with SlowAPI
To prevent brute force attacks and abuse of the authentication endpoints, we integrated slowapi to limit the number of registration and login attempts per IP address. This ensures server stability and guards against malicious actors.

5. Shared Authentication Component (Frontend)
On the frontend, we built a single shared AuthForm component for both login and signup. This kept the code DRY (Don’t Repeat Yourself), easy to maintain, and consistent in behavior and design.

6. Token-Based Session Management
Once logged in, the user’s JWT token is stored securely in localStorage, and automatically included in all backend requests requiring authentication. This allowed us to gate the admin dashboard without requiring re-login between tabs or page refreshes.

Why This Phase Matters for RAG

Everything in Retrieval-Augmented Generation relies on knowing who is asking the question and what knowledge base they’re allowed to access. Without this foundational authentication layer, RAG would be vulnerable to serving private answers to the wrong users.

Admin Isolation ensures that:

• Each admin uploads their own documents

• Their chatbot answers only from those documents

• Their leads and chat activity are not visible to anyone else

This is what turns a generic chatbot into a secure knowledge assistant built around your own business.

Phase 2 - Chatbot Engine with Retrieval-Augmented Generation (RAG)

Once user authentication and admin-specific isolation were successfully implemented, the next critical layer of the platform focused on enabling each admin to upload documents, from which a highly accurate and context-aware chatbot could generate responses. This marked the core technical backbone of the entire system - the RAG pipeline.

Objective

To build a custom GPT-powered chatbot that:

• Understands and responds using only the admin's uploaded PDFs (not general internet data),

• Supports semantic search across documents using embeddings,

• Ensures complete data isolation so no admin can query another's documents.

Key Technologies Used

• FastAPI (Backend framework)

• LangChain (RAG framework)

• Chroma DB (for local vector storage)

• OpenAI GPT-3.5 Turbo (for question answering and classification)

• FAISS (alternative vectorstore option we considered)

• PyMuPDF and pdfplumber (for PDF text extraction)

RAG Flow Breakdown

1. PDF Upload and Ingestion

Admins can upload one or more documents through their dashboard. The uploaded PDFs are parsed page by page, and the text is chunked into manageable segments (e.g., 500-character blocks) for semantic indexing.

2. Embedding Creation

Each text chunk is passed through an embedding model (in this case, OpenAI’s text-embedding-ada-002) to convert it into a high-dimensional vector. This vector captures the semantic meaning of the text, enabling precise similarity search later.

3. Storage in Chroma DB

These embeddings are stored locally using Chroma as a vector database. Each admin’s documents are stored in a separate persistent directory, ensuring:

• Easy retrieval during queries,

• Full admin isolation at the file system level.

4. Query Time Retrieval

When a user types a message into the chatbot, the input is converted into an embedding and used to search for the top-k similar chunks in the admin’s document vectorstore.

5. Answer Generation (Augmentation Step)

The retrieved chunks are fed into a GPT prompt that combines them with the user’s question, forming the context block. GPT-3.5 is then instructed to answer based only on this supplied context - ensuring factual, document-based responses.

Example Query Flow

User: What is the refund policy?  
→ Embedding created from question 
→ Top 3 relevant chunks retrieved from admin’s stored documents
→ Chunks + Question sent to GPT:    “Based on the following documents, answer the user's question...” 
→ Clean, confident answer returned 

Challenges Solved

• Security: By scoping each vectorstore to the logged-in admin, we ensure no cross-contamination of data.

• Accuracy: By anchoring the language model’s output in real, context-rich data, we avoid hallucinations or irrelevant answers.

• Scalability: Each admin’s documents are indexed independently, allowing the system to grow without performance bottlenecks.

Outcome

Admins can now interact with their own custom-trained chatbot - one that understands the nuances, policies, and terminology of their own documents without any need for coding or prompt engineering. This laid the foundation for intelligent support, onboarding assistance, and lead qualification, all powered by domain-specific knowledge.

Phase 3 - Lead Capture and Classification Logic

With the RAG-powered chatbot engine fully operational and document responses restricted to each admin's uploads, the next strategic goal was to make this chatbot actionable. In other words - it shouldn’t just answer questions. It should capture leads, qualify them, and store them securely, so that businesses using this platform could directly benefit from every chat session.

This is where we transitioned from an informative chatbot to a conversion assistant.

Objective

To build an intelligent lead capture system that:

• Automatically prompts for lead details during the conversation,

• Stores each lead under the correct admin, with full data isolation,

• Avoids spam and abuse, and

• Provides a clean frontend for each admin to view and analyze their leads.

Core Functionalities

1. Trigger-Based Lead Prompting

After the second user message (to simulate interest), the chatbot prompts the user with a gentle request:

This balances user experience with lead generation - we don’t prompt too early, but we also don’t wait too long to qualify a visitor.

2. Smart Continuation After Lead Capture

Once the user provides their details, the bot:

• Stores the name, email, message, and timestamp into the database,

• Remembers the original question they asked before the lead capture prompt,

• Automatically answers that question, ensuring a smooth experience.

No awkward restarts. Just seamless flow.

3. Email Validation

To prevent junk data, the system checks if the submitted email is:

• Properly formatted,

• Not clearly fake (like abc@abc),

• And only proceeds if the input passes basic validation.

4. Admin-Level Isolation

Every lead is tagged with the admin email (retrieved via token), ensuring that:

• Admin A only sees leads submitted through their chatbot,

• Admin B never sees leads submitted to another company.

This is baked directly into the insert_lead() function and the /api/leads route logic.

5. Leads Tab in Dashboard

The Admin Dashboard displays a real-time list of leads:

• Name

• Email

• Message (user query at the time)

• Timestamp

This gives every admin a self-contained CRM view - powered entirely by chat interactions.

Additional Anti-Spam Measures

• Math Challenge (a + b): Added on both signup and login forms to reduce bot-based signups.

• Rate Limiting: Implemented using slowapi on critical endpoints like /register and /login to prevent brute-force or abuse.

Future Plans for This Module

• Lead scoring: Auto-label important leads based on message intent or keyword matches.

• Tagging and Notes: Allow admins to label leads or leave notes manually.

• CRM Integration: Push qualified leads to external tools like HubSpot, Mailchimp, or Google Sheets.

• Follow-up Automation: Trigger email replies based on query categories (e.g., pricing, support, onboarding).

Outcome

With lead capture logic in place, our chatbot transformed from an assistant into a qualified lead generator - with every answer tied to potential business value. It now collects actionable data, maintains clean segmentation per admin, and becomes a scalable growth channel for any business using it.

Phase 4 - Admin Dashboard Features and Testing Tools

Once leads were successfully captured and stored under the right admin accounts, we shifted focus to building a powerful admin dashboard - the control center for each business using the platform.

The goal was clear: give each admin visibility, control, and testing capabilities over their AI chatbot experience. But more importantly - make it usable from Day 1, even with zero technical background.

Objective

To build a modular, secure, and functional dashboard for every admin that allows them to:

• Monitor incoming leads in real-time.

• Upload documents for chatbot training.

• Test their own chatbot directly inside the dashboard.

• View platform usage over time.

• Maintain full data isolation from other businesses.

Key Features Implemented

1. Dashboard UI Structure

We designed the frontend to include tab-based navigation:

• Leads Tab

• Usage Tab (placeholder for now)

• Upload Tab

• Chatbot Tab

Each tab was self-contained and modular, allowing future upgrades without breaking anything else.

2. Leads Tab

• Displays all leads submitted via the public chatbot.

• Admins see only their own leads.

• Table columns: Name, Email, Message, Timestamp.

• Data is fetched using a secure route protected by JWT (/api/leads).

3. Document Upload Tab

• Admins can upload PDFs that contain business information (FAQs, services, policies, etc.).

• These documents are used to train the chatbot specifically for that business.

• Uploads are handled via FastAPI and stored in admin-specific folders.

4. Chatbot Tab

An embedded chatbot widget, but exclusive to the admin. It:

• Loads only the PDFs that the current admin has uploaded.

• Follows the same logic as the public chatbot (lead capture, fallback, reset, contact detection).

• Great for testing how the bot responds before embedding it on a live website.

5. Usage Tab (Coming Soon)

This placeholder is designed to:

• Track number of chatbot messages.

• Measure leads captured over time.

• Show bandwidth/storage used.

It will serve as a mini-analytics dashboard once rolled out.

Security and Access Control

• Every tab is protected via JWT-based authentication.

• We fetch the logged-in admin's email from the token and filter all data (leads, uploads, chats) accordingly.

• Document uploads are stored in private paths tied to each admin's email (hashed/encoded).

This ensures true tenant isolation, even if multiple admins use the same platform.

Developer Experience

We built the frontend using React with Vite, and styled using Tailwind CSS for rapid iteration. On the backend, we used FastAPI, and for file storage and retrieval, we kept things modular so that switching to cloud (e.g., S3) in the future is seamless.

All testing was done locally, with console feedback and error logging during uploads, chat interaction, and lead fetches.

Future Enhancements

• Admin Customization Panel: Let admins change the bot’s welcome message, theme color, and fallback behavior.

• Lead Export: Enable CSV export of leads directly from the dashboard.

• Advanced Usage Reports: Offer weekly/monthly summaries.

• API Keys & Webhooks: So admins can integrate the chatbot into external tools or marketing platforms.

Outcome

The Admin Dashboard is no longer just a formality - it’s the engine room of the product. With live testing, instant uploads, and a view of lead activity, every business owner gets full visibility and control over their AI-powered agent. And most importantly, it’s built to scale - both in features and user count.

Phase 5 - RAG-Powered Public Chatbot Deployment

This was the phase where everything came together. All the behind-the-scenes logic, lead routing, and admin isolation were finally put into public-facing action - via a RAG-powered chatbot that could be embedded on any website.

But before diving in, here’s the core concept again:

🔍 What is RAG?

Retrieval-Augmented Generation (RAG) is a technique where an AI model (like GPT) doesn't just answer from its base training data - it also retrieves context from an external knowledge base (like uploaded PDFs) before generating a response.

This gives businesses:

• Accuracy: Answers grounded in their own data, not hallucinated content.

• Customization: Each business's bot responds using their policies, products, tone, and details.

• Control: Businesses can change responses by simply updating a document.

In our case, when a user asks a question:

1. We first search through the PDFs the admin uploaded.

2. We extract relevant chunks.

3. We pass those chunks along with the question to GPT to generate the response.

This approach transforms the chatbot from “generic assistant” to deeply contextual business rep.

Objective

Deploy a public chatbot that:

• Can be embedded on any business’s website.

• Instantly answers user questions using that business’s documents.

• Captures leads with name + email after 2 interactions.

• Maintains separate data pipelines for each admin/business.

• Includes fallbacks for non-relevant queries, and reset behavior after repeated off-topic messages.

Key Features in Deployment

1. Public Widget (ChatbotWidget.jsx)

• Embeddable React component with clean UI.

• Identifies which admin’s data to use via admin_email or admin_id.

• Works independently of the dashboard - can run on any frontend.

2. Lead Capture Logic

• Kicks in automatically after the user sends 2 messages.

• Asks for name and email.

• Validates email format before proceeding.

• Stores leads in the database, tagged with the corresponding admin.

3. RAG Answering Pipeline

• Uses LangChain to:

  • Load the correct admin’s documents.

  • Retrieve the most relevant chunks for the question.

  • Pass those chunks to GPT-3.5 Turbo.

• Returns a contextualized response in under 2 seconds in most cases.

4. Contact Detection and Fallbacks

• If user asks:

  • The bot returns the hardcoded contact info.

• If 5 irrelevant questions are asked:

  • The bot resets with a message like “Let’s get back to your questions about [business].”

5. Rate Limiting and Bot Protection

• Integrated slowapi on /chat to prevent abuse.

• Email and math challenge logic prevents bot spam.

• Optional captcha layer can be added in future.

Technical Stack Overview

Future Plans

• Custom Bot Styles per Admin: Colors, fonts, and welcome messages.

• SaaS Signup Flow: So admins can create their own chatbot without any developer help.

• Multi-language Support: Based on user query or site language.

• Chat Analytics: Show admins what users are asking most, conversion rates, etc.

Outcome

The chatbot is no longer just smart - it’s now specific. Specific to each business, specific to each document, and specific to each visitor’s query. It doesn’t just generate - it retrieves and generates, creating accurate, tailored responses that actually help users.

It’s the kind of assistant every business wishes they could afford - now available via a simple embed code.

What’s Next

This project was never just about building a chatbot.

It was about building a framework - a modular, secure, and scalable foundation that any business could adopt as their own.

Now that the RAG-powered system is live, here’s what’s coming next:

Phase 6 - Admin Dashboard v2

• Real-time chat analytics

• Editable welcome messages, bot personality, and fallback behaviors

• Custom chatbot themes (colors, button shapes, avatar, etc.)

• Option to manually reply to leads through the dashboard

Phase 7 - One-Line Embed + Auto-Onboarding

• Each admin gets a unique embed code (like <script src="..."></script>)

• Clean “Get Your Chatbot” onboarding funnel

• Automated provisioning of:

  • Admin account

  • Public chatbot page

  • Document storage

  • Lead CRM

This will take us from “functional MVP” to “scalable SaaS.”

Phase 8 - Security & Performance Upgrades

• Switch to S3 or cloud storage for documents

• Auth token expiration logic and refresh tokens

• Switch from SQLite/PostgreSQL to managed cloud DB

• Add file validation + virus scanning at upload

Long-Term Vision

We're not building a chatbot product.

We’re building the engine behind every intelligent assistant that startups will need - whether it’s customer support, investor relations, hiring, or onboarding.

Each of these use cases can be powered by RAG + lead logic + multi-tenant control - and with this foundation, we’re already halfway there.

Conclusion

In less than 3 weeks, I built a complete, full-stack RAG-powered chatbot platform that:

• Allows any business to upload their documents

• Lets visitors chat with an AI trained on those documents

• Captures and routes leads securely

• Keeps each business’s data and responses 100% isolated

And I did it without external libraries for UI kits or dashboards, coding everything from scratch - learning, failing, fixing, and iterating each day.

It’s been one of the most practical and rewarding Project52 builds so far - and the backbone of a product that can scale.

If you’re building anything similar - SaaS, AI-powered tools, or even knowledge bots for internal use - feel free to borrow from this project’s structure. That’s the point of documenting the journey.

See you in Week 20.

- Atul
Project52