# Contributing to Evidence Lab

Thank you for your interest in contributing to Evidence Lab! This document provides guidelines and instructions for contributing to the project.

## 📋 Table of Contents

- [Requirements for Acceptable Contributions](#requirements-for-acceptable-contributions)
- [Code of Conduct](#code-of-conduct)
- [Getting Started](#getting-started)
- [Development Setup](#development-setup)
- [Pre-commit Hooks](#pre-commit-hooks)
- [CLAUDE.md (AI Assistant Context)](#claudemd-ai-assistant-context)
- [Security](#security)
- [Making Changes](#making-changes)
- [Commit Guidelines](#commit-guidelines)
- [Testing](#testing)
- [Evaluation](#evaluation)
- [Performance Tests](#performance-tests)
- [GitHub Actions (CI)](#github-actions-ci)
- [Pull Request Process](#pull-request-process)
- [Issue Guidelines](#issue-guidelines)
- [Code Style](#code-style)

## ✅ Requirements for Acceptable Contributions

All contributions must meet the following requirements before they will be accepted:

1. **Pre-commit hooks pass** — install and run `pre-commit` before opening a pull request (see [Pre-commit Hooks](#pre-commit-hooks)). This is mandatory, not optional. Hooks enforce formatting, linting, type checking, complexity limits, and secret detection.

2. **Coding standards** — Python code must conform to the style enforced by our toolchain:
   - **Formatting**: `black` (line length 100)
   - **Import order**: `isort`
   - **Linting**: `flake8` (with `flake8-bugbear`, `flake8-comprehensions`, `pep8-naming`)
   - **Type annotations**: `mypy` with full Optional checking enforced (no implicit Optional)
   - **Security**: `bandit` (medium and high severity findings fail the build)
   - **Complexity**: cyclomatic complexity ≤ 10, cognitive complexity ≤ 15, maintainability index ≥ 20 per file (enforced by `scripts/quality/code_metrics.py`)

3. **Tests** — new features must include unit tests; bug fixes should include a regression test. Run `pytest tests/unit/` before submitting.

4. **No secrets or credentials** — `detect-secrets` and `gitleaks` run on every commit. Never commit API keys, tokens, or passwords.

5. **Commit messages** — follow [Conventional Commits](https://www.conventionalcommits.org/) (`feat:`, `fix:`, `docs:`, `ci:`, etc.).

6. **Pull request target** — open PRs against the current release candidate branch (`rc/vX.Y.Z`) or `main` for hotfixes, never directly to `main` for new features.

If any of these requirements are not met, CI will fail and the PR will not be merged until the issues are resolved.

## 🤝 Code of Conduct

By participating in this project, you agree to abide by our [Code of Conduct](CODE_OF_CONDUCT.md). Please treat all community members with respect and create a welcoming environment for everyone.

Please read our [Code of Conduct](CODE_OF_CONDUCT.md) before contributing to ensure a positive and inclusive community experience.

## 🚀 Getting Started

### Prerequisites

- Python 3.11 or higher
- Git
- A GitHub account

### Fork and Clone

1. Fork the repository on GitHub
2. Clone your fork locally:
   ```bash
   git clone https://github.com/YOUR-USERNAME/evidencelab-ai.git
   cd evidencelab-ai
   ```
3. Add the upstream repository:
   ```bash
   git remote add upstream https://github.com/ORIGINAL-OWNER/evidencelab-ai.git
   ```

### 💻 Development Setup

See main README.

## 🔧 Pre-commit Hooks

We use pre-commit hooks to ensure code quality and consistency. **This is required for all contributors.**
This mirrors the CI `code-quality` job. Code metrics run as part of pre-commit and
fail if any file is rated **bad** (see [Code Complexity Checks](#code-complexity-checks)).

As with most recent repositories, the code in the repo was developed with AI-assistance (mostly Claude in Cursor, with a sprinkling of GitHub Copilot). This is only possible with the use of comprehensive unit and integration tests, as well as code quality automated tests, very importantly to include code complexity validation as AI can tend to make things hard to follow and support. So please ensure pre-commit hooks are activated if using AI to code. Thanks!

### Install Pre-commit

```bash
# One time
python3.11 -m venv .venv

# Activate your virtual environment first
source .venv/bin/activate  # On macOS/Linux
# .venv\Scripts\activate    # On Windows

# Install pre-commit package (if not already installed)
pip install pre-commit

# Install the git hook scripts
pre-commit install
```

### What Pre-commit Does

Our pre-commit configuration (`.pre-commit-config.yaml`) automatically:

- **Code Formatting**: Runs `black` to format Python code
- **Import Sorting**: Runs `isort` to organize imports
- **Linting**: Runs `flake8` for code quality checks
- **Type Checking**: Runs `mypy` for static type analysis
- **Python Security**: Runs `bandit` for security vulnerability scanning
- **Dockerfile Linting**: Runs `hadolint` to check Dockerfile best practices
- **Secrets Detection**: Runs `detect-secrets` and `gitleaks` to prevent credential leaks
- **Debug Guards**: Fails on `print`/`pdb` style debug statements
- **File Hygiene**: Trailing whitespace, end-of-file, mixed line endings
- **Config Validation**: Checks syntax for YAML/JSON/TOML files
- **Merge Conflict Detection**: Prevents committing unresolved conflicts
- **Large File Prevention**: Blocks files larger than 2 MB
- **Code Metrics Gate**: Fails if any file is **bad** for cyclomatic, cognitive, or MI

### Running Pre-commit Manually

```bash
# Run on all files
pre-commit run --all-files

# Run on staged files only
pre-commit run

# Update pre-commit hooks to latest versions
pre-commit autoupdate

# Skip pre-commit for emergency commits (not recommended - violates project rules)
# Note: User rules prohibit --no-verify flag
git commit -m "your message"  # Always runs pre-commit hooks
```

### Code Complexity Checks

Use the code metrics script to measure cyclomatic complexity, cognitive
complexity, and maintainability index for `pipeline`, `utils`, and `ui`:

```bash
# Activate virtual environment
source .venv/bin/activate

# Install metrics-only dependencies
pip install -r requirements.metrics.txt

# JS/TS cognitive complexity requires Node.js + frontend deps
cd ui/frontend && npm install && cd -

# Run the metrics script
python scripts/quality/code_metrics.py --fail-on-bad
```

If frontend dependencies are missing, you can skip JS/TS cognitive complexity:

```bash
python scripts/quality/code_metrics.py --skip-js-cognitive
```

Note: CI installs Node.js and frontend dependencies so JS/TS metrics run there.

## 🤖 CLAUDE.md (AI Assistant Context)

The project includes a [`CLAUDE.md`](CLAUDE.md) file at the repository root that provides project context to AI coding assistants (Claude Code, Cursor, etc.). This file contains architecture overview, build/test/deploy commands, project rules, security protocols, code patterns, and non-obvious gotchas.

### Keeping CLAUDE.md Current

When making significant changes to the project — new subsystems, changed commands, new conventions, or architectural shifts — update `CLAUDE.md` to reflect them. The file should stay concise and actionable; it is not a replacement for full documentation but a quick-reference for AI assistants working on the codebase.

### Auditing with the CLAUDE.md Improver

You can audit and improve the `CLAUDE.md` using the [CLAUDE.md Improver skill](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/claude-md-management/skills/claude-md-improver) in Claude Code. It evaluates the file across six dimensions (commands/workflows, architecture clarity, non-obvious patterns, conciseness, currency, actionability) and proposes targeted updates:

```bash
# In Claude Code, run:
/claude-md-improver
```

The skill will discover all CLAUDE.md files in the repo, score them, produce a quality report, and suggest improvements for review before applying changes.

## 🔒 Security

Security is a critical part of our development process. See [SECURITY.md](SECURITY.md) for full details.

### Security Scanning in Pre-commit

Pre-commit hooks run the following security checks on every commit:

- **Bandit**: Python static application security testing (SAST)
- **Hadolint**: Dockerfile security and best practice linting
- **detect-secrets**: Prevents accidental credential commits
- **Gitleaks**: Enhanced secret detection with regex patterns

### Security Scanning in CI

The GitHub Actions CI pipeline includes dedicated security jobs:

- **pip-audit**: Python dependency vulnerability scanning
- **npm audit**: JavaScript dependency vulnerability scanning
- **Bandit SAST**: Comprehensive Python security analysis with JSON reports
- **Hadolint**: Dockerfile linting for all Dockerfiles
- **Gitleaks**: Repository-wide secret scanning
- **Trivy**: Container image vulnerability scanning

### Frontend Security

The frontend uses ESLint security plugins:

- **eslint-plugin-security**: Detects potential security issues in JavaScript/TypeScript
- **eslint-plugin-sonarjs**: Code quality rules that catch security anti-patterns

### Dependency Management

Dependabot is configured to automatically create PRs for:

- Python dependency updates (pip)
- JavaScript dependency updates (npm)
- Docker base image updates
- GitHub Actions version updates

### Reporting Security Issues

If you discover a security vulnerability, please report it responsibly. See [SECURITY.md](SECURITY.md) for our security policy and reporting instructions.

## 🛠️ Making Changes

### Development Workflow

1. **Create a feature branch:**
   ```bash
   git checkout -b feat/your-feature-name
   ```

2. **Make your changes:**
   - Write code following our [Code Style](#code-style)
   - Add tests for new functionality
   - Update documentation as needed

3. **Test your changes:**
   - Run the Docker-based test commands in [Testing](#testing)

4. **Commit your changes** (see [Commit Guidelines](#commit-guidelines))

5. **Push and create PR:**
   ```bash
   git push origin feat/your-feature-name
   ```

### Types of Contributions

- **Bug Fixes**: Fix existing issues
- **Features**: Add new functionality
- **Documentation**: Improve docs, examples, or comments
- **Tests**: Add or improve test coverage
- **Performance**: Optimize existing code
- **Refactoring**: Improve code structure without changing functionality

## 📝 Commit Guidelines

We use [Conventional Commits](https://www.conventionalcommits.org/) format:

### Commit Format

```
<type>(<scope>): <description>

[optional body]

[optional footer(s)]
```

### Commit Types

- **feat:** New features
- **fix:** Bug fixes
- **docs:** Documentation changes
- **style:** Code style changes (formatting, etc.)
- **refactor:** Code refactoring
- **test:** Adding or modifying tests
- **chore:** Build process or auxiliary tool changes
- **perf:** Performance improvements
- **ci:** CI/CD pipeline changes
- **build:** Build system or dependencies

### Examples

```bash
feat: add language detection to parse pipeline
fix: resolve PDF parsing error for large documents
docs: update README with metadata preparation examples
test: add unit tests for summarization module
chore: update dependencies to latest versions
```

### Commit Best Practices

- Keep commits atomic (one logical change per commit)
- Write clear, descriptive commit messages
- Reference issues when applicable: `fixes #123`
- Use imperative mood: "add feature" not "added feature"

## 🧪 Testing

### Run Tests (Docker)

```bash
# Start services (required for integration and UI tests)
docker compose up -d

# Run unit tests (fast)
docker compose exec pipeline pytest tests/unit/ -v

# Run frontend unit tests (React)
docker compose exec -e CI=true ui npm test -- --watchAll=false

# Run integration tests (requires API + UI containers)
# The ingest a document, and run a webbrowser to test end-to-end behavior
# Note: This can be slow in docker, see also ./tests/integration/run_integration_host_pipeline.sh for
# and example of running on host. You may need to tune this for your environment.
API_BASE_URL=http://api:8000 UI_BASE_URL=http://ui:3000 \
  docker compose exec pipeline pytest tests/integration/ -v -s

# Run all tests (pipeline container)
docker compose exec pipeline pytest -v
```

For additional test details, see `tests/README.md`.

### Manual Verification (UI)

When changes affect parsing, chunking, or rendering, validate in the UI:

1. Re-index a document and open it in the UI.
2. Search for a query that returns figures/tables.
3. Confirm images, tables, and inline references render in the right order.

## 🔎 Evaluation

Evaluation scripts call live LLMs and the running API. Expect latency and
token costs.

### Search Evaluation

```bash
# Generate a small synthetic dataset
docker compose exec pipeline \
  python tests/evaluation/search/generate_search_tests.py --num-queries 5

# Run evaluation (default model from env)
docker compose exec pipeline python tests/evaluation/search/evaluate.py

# Specify embedding model and rerank
docker compose exec pipeline \
  python tests/evaluation/search/evaluate.py --model e5_large --rerank
```

### TOC Classification Evaluation

```bash
# LLM-judge TOC hierarchy for multiple documents
docker compose exec pipeline \
  python tests/evaluation/toc_classification/test_toc_hierarchy.py --records 5

# Evaluate a specific document (optionally reparse)
docker compose exec pipeline \
  python tests/evaluation/toc_classification/predict_one_toc_all.py --file-id <file_id> --data-source uneg

# Bulk tag prediction from a CSV file
docker compose exec pipeline \
  python tests/evaluation/toc_classification/predict_toc_tags.py <input_csv> <output_csv>
```

## ⚡ Performance Tests

Performance scripts live in `scripts/performance/` and expect the Docker stack
to be running.

```bash
# Quick API latency check across model combos
docker compose exec pipeline \
  python scripts/performance/verify_search_performance.py

# Qdrant hybrid query diagnostics
docker compose exec -e QDRANT_HOST=http://qdrant:6333 pipeline \
  python scripts/performance/verify_qdrant_performance.py
```

### Search stress test

`scripts/performance/search-stress-test.py` fires randomised queries at the
`/api/search` endpoint and prints results asynchronously as they arrive.

```bash
# Against a remote deployment (e.g. production)
python scripts/performance/search-stress-test.py \
  --base-url https://evidencelab.ai \
  --api-key "$API_KEY" \
  --data-source "UN Humanitarian Evaluation Reports" \
  --model-combo "Huggingface" \
  --total-requests 20 --pause 1

# Against local Docker stack, compare rerank on/off
docker compose exec pipeline \
  python scripts/performance/search-stress-test.py \
    --base-url http://api:8000 --search-path /search \
    --data-source uneg --run-both
```

Key options:

| Flag | Default | Description |
|------|---------|-------------|
| `--model-combo` | *(none)* | Load model + reranker from `config.json` (`Azure Foundry`, `Huggingface`) |
| `--total-requests` | 300 | Number of queries to send |
| `--pause` | 1.0 | Seconds between sends (0 = fully concurrent) |
| `--concurrency` | 50 | Max parallel requests (used when `--pause 0`) |
| `--run-both` | off | Run once with rerank, once without |
| `--rerank-model-page-size` | 10 | Candidates per rerank batch (critical for HF reranker) |
| `--output-json` | *(none)* | Write results to a JSON file |

## ✅ GitHub Actions (CI)

The CI workflow in `.github/workflows/ci.yml` runs on push and PRs:

- **code-quality**: Pre-commit hooks (formatting, linting, type checks, security scans).
- **security-scan**: Dependency audits (pip-audit, npm audit), Bandit SAST, Hadolint, Gitleaks.
- **container-scan**: Trivy vulnerability scanning on Docker images.
- **tests**: Python unit tests (`pytest tests/unit/`).
- **integration-tests**: Docker Compose stack + integration tests.
- **ui-tests**: React unit tests (`npm test -- --watchAll=false`).

Secrets and environment variables for integrations are configured in GitHub
Actions (`HUGGINGFACE_API_KEY`, API URLs, etc.).

### Dependabot

Dependabot is configured (`.github/dependabot.yml`) to automatically create weekly PRs for:

- Python dependencies (`requirements.txt`)
- NPM dependencies (`ui/frontend/package.json`)
- Docker base images
- GitHub Actions versions

## 🔍 Pull Request Process

### Before Submitting

1. **Update your branch:**
   ```bash
   git fetch upstream
   git rebase upstream/main
   ```

2. **Run all checks:**
   - Run the Docker-based test commands in [Testing](#testing)

3. **Update documentation** if needed

### PR Guidelines

- **Title**: Use conventional commit format
- **Description**: Clearly explain what and why
- **Link Issues**: Reference related issues
- **Screenshots**: Include for UI changes
- **Breaking Changes**: Clearly document any breaking changes

### PR Template

```markdown
## Description
Brief description of changes

## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Other: ___

## Testing
- [ ] Tests pass locally
- [ ] New tests added for new functionality
- [ ] Manual testing completed

## Checklist
- [ ] Code follows project style guidelines
- [ ] Self-review of code completed
- [ ] Documentation updated
- [ ] No breaking changes (or clearly documented)
```

### Review Process

1. **Automated Checks**: CI/CD pipeline runs tests and linting
2. **Peer Review**: At least one maintainer reviews the code
3. **Feedback**: Address any requested changes
4. **Approval**: Maintainer approves and merges

## 🐛 Issue Guidelines

### Before Creating an Issue

1. **Search existing issues** to avoid duplicates
2. **Check documentation** for common solutions
3. **Test with latest version**

### Bug Reports

Include:
- **Environment**: OS, Python version, package versions
- **Steps to reproduce** the issue
- **Expected behavior**
- **Actual behavior**
- **Error messages** (full stack trace)
- **Minimal code example**
- **Sample metadata file** (if related to parsing/summarization)

### Feature Requests

Include:
- **Clear description** of the feature
- **Use case** and motivation
- **Proposed solution** (if you have one)
- **Alternatives considered**

## 🎨 Code Style

### Python Style Guide

- **Formatter**: Black (line length: 88 characters)
- **Import Sorting**: isort (Black-compatible profile)
- **Linting**: flake8 with extensions for complexity and security
- **Type Checking**: mypy (Optional checking enforced; scripts/ and alembic/ excluded)
- **Type Hints**: Use type hints for function signatures
- **Docstrings**: Google style docstrings

### Code Quality Requirements

1. **Type Hints**: All functions must have type hints
2. **Docstrings**: All public functions and classes must have docstrings
3. **Test Coverage**: Minimum 90% test coverage for new code
4. **Security**: No security vulnerabilities (checked by Bandit)
5. **Complexity**: Keep cyclomatic complexity under 10

### Code Quality Commands

```bash
# Activate virtual environment
source .venv/bin/activate

# Format code
black pipeline/ tests/
isort pipeline/ tests/

# Check code quality
flake8 pipeline/ tests/
mypy --ignore-missing-imports pipeline/
```

## 🚀 Adding New Features

### Process

1. **New Pipeline Scripts**: Add new pipeline components in the `pipeline/` directory
2. **Include proper type hints and documentation** for all functions
3. **Add corresponding tests** in `tests/`
4. **Update the README** with documentation for your feature
5. **Update metadata columns** if your feature adds new data to the Excel file

### Extending the Pipeline

New pipeline stages should:
- Read from and write to the metadata Excel file
- Follow the existing pattern of parse → summarize → stats
- Include error handling and logging
- Support command-line arguments for flexibility

## 🔄 Release Process

We use **release candidate branches** (`rc/vX.Y.Z`) to stage and stabilise
changes before they reach `main`.

### Branch Strategy

```
feature/fix branches ──► rc/vX.Y.Z ──► main
                           (staging)     (stable)
```

| Branch | Purpose |
|--------|---------|
| `main` | Stable, production-ready code. Only receives merges from RC branches. |
| `rc/vX.Y.Z` | Release candidate. All feature and fix PRs target this branch. |
| `feat/*`, `fix/*`, etc. | Short-lived branches for individual changes. |

### Workflow

1. **Create an RC branch** from `main` when starting a new release cycle:
   ```bash
   git checkout main
   git checkout -b rc/v1.2.0
   git push -u origin rc/v1.2.0
   ```

2. **Target PRs to the RC branch.** All feature and fix PRs should set
   `rc/vX.Y.Z` as their base branch, not `main`:
   ```bash
   gh pr create --base rc/v1.2.0
   ```

3. **Retarget existing PRs** if switching to a new RC branch:
   ```bash
   gh pr edit <PR_NUMBER> --base rc/v1.2.0
   ```

4. **Dependabot PRs** also target the RC branch (configured via
   `target-branch` in `.github/dependabot.yml`). Update this value
   when creating a new RC branch.

5. **Merge RC to main** once all PRs are merged and the release is validated:
   ```bash
   git checkout main
   git merge rc/v1.2.0
   git tag v1.2.0
   git push origin main --tags
   ```

6. **Create a GitHub Release** from the tag with release notes.

### Release Checklist

1. All PRs merged into `rc/vX.Y.Z`
2. CI passing on the RC branch
3. Manual verification on staging/Docker environment
4. Version bump in `pyproject.toml`
5. Merge RC branch to `main`
6. Tag release and create GitHub Release with notes

## 📞 Getting Help

- **Documentation**: Check the README and code comments
- **Issues**: Create a GitHub issue for bugs or feature requests
- **Discussions**: Use GitHub Discussions for questions
- **UNEG Resources**: Consult [UNEG Evaluation Reports](https://www.unevaluation.org/)

## 🙏 Recognition

Contributors will be recognized in:
- GitHub contributors list
- Release notes for significant contributions
- Project acknowledgments

Thank you for contributing to Evidence Lab! 🎉
