Testing

The platform uses pytest with async support for comprehensive testing.

Test Structure

backend/tests/
├── conftest.py                    # Shared fixtures
├── test_diagnosis_to_project_fix.py
├── test_form_templates_phase4.py
├── test_form_trigger_template_resolution.py
├── test_invite_join.py
├── test_members.py
├── test_notifications_collab.py
├── test_permission_guards.py
├── test_project_maturity.py
├── test_roles_enums.py
├── agents/                        # Agent-specific tests
├── api/                           # API endpoint tests
├── integration/                   # Integration tests
├── llm/                           # LLM evaluation tests
├── load/                          # Load/performance tests
└── unit/                          # Unit tests

Running Tests

cd backend

# Run all tests
pytest

# Run with verbose output
pytest -v

# Run a specific test file
pytest backend/tests/test_invite_join.py

# Run tests matching a pattern
pytest -k "test_permission"

# Run with coverage
pytest --cov=backend --cov-report=html

Test Categories

Unit Tests (tests/unit/):

Test individual services and repositories in isolation with mocked dependencies.

API Tests (tests/api/):

Test HTTP endpoints using FastAPI’s TestClient with a test database.

Integration Tests (tests/integration/):

Test full workflows across multiple services with real database connections.

Agent Tests (tests/agents/):

Test AI agent behavior with mocked LLM responses.

LLM Evaluation Tests (tests/llm/):

Use DeepEval for measuring response quality, relevance, and safety.

Load Tests (tests/load/):

Use Locust for performance testing under concurrent load.

Key Fixtures (conftest.py)

@pytest.fixture
async def db_session():
    """Provides a test database session with rollback."""
    ...

@pytest.fixture
async def test_client():
    """FastAPI test client with overridden dependencies."""
    ...

@pytest.fixture
def auth_headers():
    """Authentication headers for test requests."""
    return {"X-User-Id": "test-user", "X-User-Name": "Test User"}

Load Testing with Locust

# Start Locust
cd backend
locust -f backend/tests/load/locustfile.py

# Open Locust UI
open http://localhost:8089

Configuration:

  • Target: backend API URL

  • Users: Configurable concurrent users

  • Spawn rate: Users per second

LLM Evaluation with DeepEval

# Run LLM evaluation tests
pytest backend/tests/llm/ -v

Metrics evaluated:

  • Answer relevancy — Is the response relevant to the question?

  • Faithfulness — Is the response grounded in provided context?

  • Toxicity — Is the response free from harmful content?

  • Hallucination — Does the response contain fabricated facts?