客户支持聊天机器人

📦 Prerequisites

2. Install dependencies

pip install -r requirements.txt python -m nltk.downloader punkt stopwords

Best quality (requires Cohere API key)

COHERE_API_KEY=your-key # → CohereReranker auto-selected

Local cross-encoder (no API, requires GPU/CPU)

LoRA — full precision, requires more VRAM

python scripts/finetune_adapter.py \ --method lora \ --lora-r 32 \ --epochs 5 ```

E2E tests (requires running app)

make up pytest tests/e2e/ --base-url http://localhost:8000 -v ```

All checks (required before PR)

make format lint test ```

Project 2 - Build a Customer Support Chatbot using RAGs and Prompt Engineering

Option A: Docker (Recommended — Full Stack)

```bash

or: docker-compose up -d

Option B: Local Development (No Docker)

```bash

Verify Installation

```bash

Local development (default, no setup needed)

VECTOR_STORE_TYPE=chroma

Rebuild Index

```bash

Rebuild the entire index from processed JSONL files

python scripts/build_index.py \ --processed-dir ./data/processed \ --store-type chroma \ --embedding-model text-embedding-3-small \ --clear # wipe and rebuild from scratch ``` ---

🚢 Deployment

Deploy with Docker Compose

```bash

Production deployment

docker-compose -f docker-compose.yml up -d

5. Enable Redis caching (already in docker-compose)

REDIS_URL=redis://localhost:6379/0

For Docker:

docker-compose restart redis

Development Setup

git clone https://github.com/yourorg/customer-support-chatbot.git
cd customer-support-chatbot
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
pre-commit install

🚀 Quick Start

Chunking Strategy Guide

Strategy         When to Use                          Trade-off
─────────────────────────────────────────────────────────────────────
fixed_size       Simple docs, uniform content         Fast, may split mid-sentence
sentence         Conversational content, FAQs         Better coherence, slower
paragraph        Policy docs, structured text         Natural boundaries
hierarchical     Long docs with headings ← recommended Best recall + context
semantic         Mixed content, research docs         Best coherence, needs GPU
sliding_window   Dense technical docs                 High recall, more chunks

LoRA Rank Selection Guide

rank (r)   Parameters Added    Quality    Speed
────────────────────────────────────────────────
4          ~1M                 Good       Fastest
8          ~2M                 Better     Fast
16         ~4M   ← recommended Excellent  Good
32         ~8M                 Best       Slower
64         ~16M                Marginal+  Slowest

Pull Request Guidelines

1. Branch naming: feature/, fix/, docs/, refactor/ 2. Commit messages: Follow Conventional Commits

   feat: add semantic chunking strategy
   fix: resolve FAISS persistence bug
   docs: update chunking strategy guide
   

5. Update this README under Chunking Strategy Guide

```

4. Add API key to settings.py and .env.example

4. Add sample documents and ingest

Sample Report Output

════════════════════════════════════════════════════════════
  RAG EVALUATION REPORT: production_eval
════════════════════════════════════════════════════════════
  Total Samples:     150
  Overall Pass Rate: 87.3%
  Evaluation Time:   243.1s

  Metric Scores:
  ──────────────────────────────────────────────────────────
  ✅ faithfulness               avg=0.891 ± 0.087  pass=91.3%
  ✅ answer_relevance           avg=0.873 ± 0.094  pass=88.7%
  ✅ context_relevance          avg=0.812 ± 0.121  pass=84.0%
  ✅ answer_correctness         avg=0.798 ± 0.132  pass=82.0%
  ⚠️  context_recall            avg=0.681 ± 0.156  pass=71.3%
  ✅ rouge                      avg=0.412 ± 0.189  pass=79.3%
  ✅ citation_precision         avg=0.834 ± 0.097  pass=87.3%
  ✅ answer_length              avg=0.943 ± 0.067  pass=94.7%
════════════════════════════════════════════════════════════

Optional External Services

| Service | Default | Alternative | |---|---|---| | Vector Store | ChromaDB (local) | Pinecone (cloud) or FAISS (disk) | | Keyword Search | BM25 (in-memory) | Elasticsearch (Docker) | | Conversation Store | Redis (Docker) | In-memory fallback | | Database | PostgreSQL (Docker) | SQLite for dev | ---

2. Configure environment

cp .env.example .env

Edit .env — minimum required:

1. Clone and set up Python environment

git clone https://github.com/AdilShamim8/Customer-Support-Chatbot-102.git cd customer-support-chatbot python -m venv venv source venv/bin/activate # Windows: venv\Scripts\activate

3. Configure environment

cp .env.example .env

Set OPENAI_API_KEY and APP_ENV=development

nano .env

⚙️ Configuration

All configuration is via environment variables. Copy .env.example to .env and fill in your values.

Core Settings

```env

In .env — point to local merged model

LLM_PROVIDER=local CHAT_MODEL=./models/support-lora-merged ``` ---

Add to .env for production:

SECRET_KEY=your-secret-key

1. Set production environment variables

APP_ENV=production LOG_LEVEL=INFO SECRET_KEY=<cryptographically-random-32-bytes> API_RATE_LIMIT=60

Check .env file exists and has the key

cat .env | grep OPENAI_API_KEY

Reload environment

source .env

#### `ChromaDB connection refused`

1. Add to LLMProvider enum in settings.py

class LLMProvider(str, Enum): MY_PROVIDER = "my_provider"

API Keys (at least one LLM provider required)

| Service | Required | Purpose | |---|---|---| | OpenAI | ✅ Recommended | GPT-4o for generation + embeddings | | Anthropic | Optional | Claude 3 alternative | | Cohere | Optional | Best-in-class reranking |

OPENAI_API_KEY=sk-...

nano .env

6. Test the API

curl -X POST http://localhost:8000/api/v1/chat/ \ -H "Content-Type: application/json" \ -d '{"message": "What is your return policy?"}'

7. Open API docs

open http://localhost:8000/docs

5. Start the API server

make dev

or: uvicorn src.api.main:app --reload --port 8000

Set COHERE_API_KEY empty → FlashRankReranker auto-selected

``` ---

Via API (Upload)

```bash

📡 API Reference

CLI evaluation against test dataset

python scripts/evaluate_pipeline.py \ --dataset ./data/eval/test_set.jsonl \ --output ./data/eval/report.json \ --max-samples 100

Without LLM metrics (fast, no API calls)

python scripts/evaluate_pipeline.py \ --no-llm-metrics

Requests by endpoint

sum by (handler)(rate(http_requests_total{job="chatbot"}[5m])) ```

4. Disable debug endpoints

Scale API workers

docker-compose up -d --scale chatbot=3

(no API call, runs locally)

COHERE_API_KEY= # leave empty → FlashRank selected

In src/api/main.py, set:

check_hallucination=True

🔬 Research & References

This system implements techniques from the following papers: | Paper | Application | |---|---| | RAFT: Adapting Language Model to Domain Specific RAG | Fine-tuning strategy | | Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks | RAG foundation | | Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods | Hybrid search fusion | | LoRA: Low-Rank Adaptation of Large Language Models | Parameter-efficient fine-tuning | | QLoRA: Efficient Finetuning of Quantized LLMs | Memory-efficient training | | RAGAS: Automated Evaluation of RAG | Evaluation framework | | Lost in the Middle | Context positioning insight | ---

Document Ingestion Pipeline

Raw Documents (PDF, DOCX, HTML, MD, TXT)
         │
         ▼
┌─────────────────────┐
│   DOCUMENT PARSER   │
│  Rule-based         │
│  AI-based (unstr.)  │
│  → ParsedDocument   │
└────────┬────────────┘
         │
         ▼
┌─────────────────────┐
│      CHUNKER        │
│  Fixed / Sentence   │
│  Paragraph          │
│  Hierarchical  ←── recommended
│  Semantic           │
│  Sliding Window     │
│  → DocumentChunk[]  │
└────────┬────────────┘
         │
         ├──────────────────────────────────┐
         ▼                                  ▼
┌────────────────────┐           ┌────────────────────┐
│   VECTOR INDEX     │           │   KEYWORD INDEX    │
│  Embed chunks      │           │  Tokenize chunks   │
│  ChromaDB / FAISS  │           │  BM25 / Elastic    │
│  / Pinecone        │           │                    │
└────────────────────┘           └────────────────────┘
         │                                  │
         └──────────────┬───────────────────┘
                        ▼
                  HYBRID INDEX
                (Ready for search)

🔄 RAG Pipeline

`GET /admin/stats` — Pipeline Statistics

curl http://localhost:8000/api/v1/admin/stats

{
  "vector_store_count": 1842,
  "keyword_index_count": 1842,
  "document_count": 47,
  "llm_requests": 1293,
  "total_tokens_used": 2847291,
  "total_cost_usd": 14.23
}

Integration tests (mocked LLM)

make test-integration

Integration test pattern (mock LLM)

@pytest.mark.asyncio async def test_pipeline(mock_pipeline): response = await mock_pipeline.aquery("How do I reset my password?") assert response.answer assert len(response.sources) > 0

🔧 Troubleshooting