Testing Strategy¶

Overview¶

ADEPT employs a comprehensive testing strategy built on the principle of zero-mock testing -- preferring real framework components over simulated responses. All tests execute inside Docker containers to ensure environment consistency across development machines and CI.

Testing Philosophy¶

Core tenets:

• Real services -- tests interact with actual Keycloak, PostgreSQL, Redis, and MCP servers.
• State isolation -- unique thread IDs and session identifiers prevent cross-test contamination.
• Container execution -- all tests run inside the agentic-framework-deps-base image with pre-installed dependencies.
• Deterministic fixtures -- test data is created per-test and cleaned up after execution.

Test Tiers¶

Container Execution¶

All tests execute inside Docker containers using a dual-network pattern:

docker run --rm \
  --network=adept_application_network \
  --network=adept_frontend_network \
  -v "$(pwd)/src":/app/src:ro \
  -v "$(pwd)/tests":/app/tests:ro \
  agentic-framework-deps-base \
  /app/.venv/bin/pytest tests/e2e/test_example.py -v

Key conventions:

• Source and test mounts are always read-only (:ro).
• The container's pre-installed /app/.venv is used directly (no ephemeral venv creation).
• Dual networks provide access to both internal services and the reverse proxy.

Contract Testing¶

Contract tests validate that class interfaces remain stable by checking:

1. Attribute existence -- all instance attributes referenced in methods exist after __init__.
2. Method execution -- public methods execute without AttributeError given typical inputs.
3. Negative assertions -- incorrect attributes (e.g., self.logger when a module-level logger is used) do not exist.

class TestMyClassContract:
    def test_init_creates_required_attributes(self, instance):
        assert hasattr(instance, 'config')
        assert hasattr(instance, 'session_id')
        # Negative: ensure no incorrect attributes
        assert not hasattr(instance, 'logger')

    def test_method_executes_without_crash(self, instance):
        try:
            result = instance.process(sample_input)
            assert result is not None
        except AttributeError as e:
            pytest.fail(f"Contract violation: {e}")

Contract tests are added for any class exceeding 500 lines or after discovering an AttributeError bug.

Test Organization¶

tests/
├── unit/                    # Component-level, no external deps
│   ├── agentic_framework_pkg/
│   └── agentic_framework_sdk/
├── e2e/                     # Full stack, real services
├── integration/             # Module interactions with stubs
└── performance/             # Load and stress tests

Coverage Goals¶

Coverage is tracked via pytest-cov and reported in CI. The make validate-unit-coverage target generates HTML coverage reports for local inspection.