Architecture for Testable Code (Part 2: State & Infrastructure)

In Part 1: The Core Patterns of this series, we fixed the code. We injected dependencies, isolated business logic, and made our functions strictly typed and predictable.

But clean code is only half the battle. You can write the most beautiful, testable functions in the world, and your test suite will still crash randomly if Test A leaks database state into Test B, or if your CI pipeline shares a staging database with three other active pull requests.

Part 2 is about the system level. The following patterns cover how to architect your state, environment configurations, and CI infrastructure so your tests are actually trustworthy.

1. Isolate Test State (Don't Let Tests Leak)

One of the most frustrating experiences in software engineering is a test that passes when run by itself, but fails when you run the entire suite. This is almost always caused by "State Leakage."

State leakage happens when Test A modifies a global variable, database row, or framework configuration, and fails to clean it up before Test B runs. Test B then inherits a mutated environment and crashes mysteriously.

Every test must be an isolated island. It should set up its own state, execute, and then burn the island down before the next test begins.

The Anti-Pattern: Leaking Framework State

In FastAPI, replacing dependencies for testing is incredibly powerful. However, if you apply an override globally and forget to remove it, every subsequent test is forced to use that fake dependency.

The Leaky Test:

# tests/test_auth.py
from app import app
from routes.dependencies import verify_token

def test_admin_dashboard():
    # BAD: We override the auth token to simulate an admin...
    app.dependency_overrides[verify_token] = lambda: {"role": "admin"}
    
    response = client.get("/admin/dashboard")
    assert response.status_code == 200
    
    # ...but we forget to clean it up! 
    # Every test that runs after this one is now permanently an Admin.

The Production Standard:

Guaranteed Teardown with Fixtures Pytest fixtures using the yield keyword elegantly solve this problem. The code before the yield sets up the environment, the test runs, and the code after the yield is guaranteed to execute, tearing down the state even if the test fails.

The Clean Test:

# tests/test_auth.py
import pytest
from app import app
from routes.dependencies import verify_token

@pytest.fixture
def mock_admin():
    # 1. SETUP: Override the dependency
    app.dependency_overrides[verify_token] = lambda: {"role": "admin"}
    
    # 2. YIELD: Hand control over to the test
    yield 
    
    # 3. TEARDOWN: Guaranteed to run, restoring the app to its original state
    app.dependency_overrides.clear()

def test_admin_dashboard(mock_admin):
    # GOOD: The test only cares about its logic. 
    # The fixture handles the lifecycle.
    response = client.get("/admin/dashboard")
    assert response.status_code == 200

By strictly managing setup and teardown, you guarantee that your test suite is deterministic: if it fails, it failed because the code is broken, not because of a ghost from a previous test.

2. Idempotent Operations Eliminate Fixture Complexity

An operation is "idempotent" if running it once produces the exact same result as running it one hundred times.

Operations that produce different results depending on what ran before them require complex test setup and teardown to isolate properly. If an endpoint fails violently because a database record already exists (or creates duplicate garbage data), your test suite must meticulously wipe and rebuild the database for every single test.

Making operations explicitly handle both cases—gracefully ignoring duplicates or performing safe "upserts"—drastically reduces the number of database resets you have to track.

The Anti-Pattern: State-Dependent Operations

This bookmarking function blindly appends data. If it runs twice, it corrupts the database with duplicate IDs. To test this safely, the test must guarantee the database is 100% empty before it runs.

The App Logic:

def add_favorite_insight(db_session, user_id: str, insight_id: int):
    user = db_session.query(User).filter_by(id=user_id).first()
    
    # BAD: Blindly appending. If called twice, it creates duplicates [42, 42]
    user.favourite_insights.append(insight_id)
    
    db_session.commit()
    return user

The Ugly Test:

def upsert_user_profile(db_session, user_id: int, bio: str):
    # Idempotent: entirely safe to run repeatedly in the same test context
    profile = db_session.query(Profile).filter_by(user_id=user_id).first()
    if not profile:
        profile = Profile(user_id=user_id)
        db_session.add(profile)
        
    profile.bio = bio
    db_session.commit()
    return profile

The Production Standard: Idempotent Operations

By adding a simple state check, the operation becomes bulletproof. It is entirely safe to run repeatedly in the same test context, meaning you don't have to stress about database cleanup if a previous test left some data behind.

The App Logic:

def add_favorite_insight(db_session, user_id: str, insight_id: int):
    user = db_session.query(User).filter_by(id=user_id).first()
    
    # GOOD: Idempotent. Safely ignores the action if it's already done.
    if insight_id not in user.favourite_insights:
        # We create a new list to ensure SQLAlchemy detects the change
        user.favourite_insights = user.favourite_insights + [insight_id]
        db_session.commit()
        
    return user

The Clean Test:

def test_add_favorite_insight_is_idempotent():
    # GOOD: We can spam the function as many times as we want.
    # We don't care what state the database was in before this test started.
    add_favorite_insight(db, "u1", 42)
    add_favorite_insight(db, "u1", 42)
    add_favorite_insight(db, "u1", 42) 
    
    user = db.query(User).filter_by(id="u1").first()
    
    # The result is perfectly consistent and predictable.
    assert len(user.favourite_insights) == 1

When your functions are self-healing and idempotent, your test suite stops crashing due to accidental data leftovers.

3. Environment Variable Injection (Decoupling Configuration)

Sprinkling os.getenv() calls deep inside business logic hides dependencies and destroys type safety. Because environment variables are always strings, your core logic is forced to manually cast booleans, integers, and lists at the point of use.

Worse, reading from the OS directly turns your test suite into a minefield. To test these functions, you are forced to mutate the global os.environ dictionary. This creates a severe global state leak hazard: if a test crashes before it cleans up the injected variable, every subsequent test inherits a corrupted environment. Furthermore, because os.environ is a single shared state across the entire Python process, running your tests in parallel (using tools like pytest-xdist) will cause race conditions where tests overwrite each other's variables, leading to flaky, impossible-to-debug failures.

Instead of reading environment variables at the point of use, load them once at application startup into a central, strictly typed configuration object (using tools like Pydantic's BaseSettings) and inject that object into your services. This gives you instant type validation, explicit dependencies, and perfectly isolated tests.

The Anti-Pattern: Hidden OS Lookups

The function secretly depends on the host environment and lacks type safety. Tests must manually mutate global state, risking leaks and breaking parallel test execution entirely.

The App Logic:

import os

def connect_to_payment_gateway(payload: dict) -> bool:
    # BAD: Hidden dependency, and zero type safety.
    # 'timeout' is returned as a string, forcing manual casting downstream.
    api_key = os.getenv("STRIPE_API_KEY")
    timeout = os.getenv("GATEWAY_TIMEOUT", "30") 
    
    if not api_key:
        raise ValueError("Missing API key")
        
    return True

The Ugly Test:

import os
from services.payments import connect_to_payment_gateway

def test_payment_gateway_success():
    # BAD: Mutating global OS state. If tests run in parallel, another 
    # test might overwrite or delete this key at the exact same millisecond.
    os.environ["STRIPE_API_KEY"] = "fake_test_key"
    os.environ["GATEWAY_TIMEOUT"] = "10"
    
    result = connect_to_payment_gateway({"amount": 100})
    assert result is True
    
    # BAD: If the assertion above fails, this line never runs. 
    # The fake key leaks into every test that runs after this one.
    del os.environ["STRIPE_API_KEY"]
    del os.environ["GATEWAY_TIMEOUT"]

The Production Standard: Typed Config Injection

The function accepts a strongly typed configuration object. The application layer handles reading and casting the environment variables automatically on startup, while the test layer simply injects a safe, dummy Python object.

The App Logic:

from pydantic_settings import BaseSettings

# 1. Define the exact shape and types of your config.
# BaseSettings automatically checks the OS environment for matching names.
class AppSettings(BaseSettings):
    stripe_api_key: str
    gateway_timeout: int = 30  # Pydantic will auto-cast OS strings to ints

def connect_to_payment_gateway(payload: dict, config: AppSettings) -> bool:
    # GOOD: Configuration is explicitly injected and strongly typed.
    # We know config.gateway_timeout is safely an integer.
    if not config.stripe_api_key:
        raise ValueError("Missing API key")
        
    return True

The Clean Test:

from services.payments import connect_to_payment_gateway, AppSettings

def test_payment_gateway_success():
    # GOOD: We pass an isolated, strongly typed Python object. 
    # No OS mutation, zero global leaks, and 100% safe for parallel test execution.
    fake_config = AppSettings(stripe_api_key="fake_test_key", gateway_timeout=10)
    
    result = connect_to_payment_gateway({"amount": 100}, config=fake_config)
    assert result is True

4. Ephemeral Infrastructure for E2E Tests (Stop Polluting Staging)

When running end-to-end (E2E) tests with tools like Playwright or Cypress, a common mistake is pointing the CI pipeline at a shared "staging" or "dev" database and Redis instance. This guarantees a nightmare of cross-pipeline collisions the moment two developers push code at the same time.

If the CI run for Pull Request A runs a teardown script to wipe the database for a clean slate, while the CI run for Pull Request B is right in the middle of asserting a user's existence, Pull Request B will inexplicably crash. Furthermore, if a developer cancels a CI run mid-execution, the teardown scripts never run, slowly filling your shared staging environment with thousands of corrupted, ghost test records.

Instead of polluting shared environments, your CI pipeline should spin up completely isolated, ephemeral Docker containers for your database and Redis. The application boots up, connects to these blank-slate containers, runs its migrations, executes the E2E tests, and then instantly destroys the containers. Every single PR gets a pristine, entirely private environment.

The Anti-Pattern: Shared Staging Infrastructure

The CI workflow runs the E2E tests against an external, long-lived staging database. Parallel CI runs actively destroy each other's state, and test data permanently leaks into the shared environment.

The Ugly CI Workflow:

# BAD: Pointing CI E2E tests to a permanent staging environment
name: E2E Tests
steps:
  - run: npm install
  - run: npx playwright test
    env:
      # These credentials point to a shared staging DB. 
      # If two PRs run tests at the same time, they will corrupt each other's data.
      DATABASE_URL: "postgres://user:pass@staging-db.company.com/staging"
      REDIS_URL: "redis://staging-redis.company.com:6379"

The Production Standard: Ephemeral CI Containers

Modern CI providers (like GitHub Actions) allow you to define services. These are disposable Docker containers that boot up alongside your test runner. Your tests interact with a real database and Redis, but it is 100% isolated to that specific CI run.

The Clean CI Workflow:

# GOOD: Ephemeral infrastructure spun up exclusively for this specific CI run.
name: E2E Tests
jobs:
  playwright-tests:
    runs-on: ubuntu-latest
    
    # 1. Spin up blank-slate containers for Postgres and Redis
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_USER: test_user
          POSTGRES_PASSWORD: test_pass
          POSTGRES_DB: test_db
        ports:
          - 5432:5432
      redis:
        image: redis:7
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      
      # 2. Run migrations to build the schema in the fresh database
      - run: npm run db:migrate
        env:
          DATABASE_URL: "postgres://test_user:test_pass@localhost:5432/test_db"
          
      # 3. Run tests against the isolated infrastructure
      - run: npx playwright test
        env:
          DATABASE_URL: "postgres://test_user:test_pass@localhost:5432/test_db"
          REDIS_URL: "redis://localhost:6379"
          
      # 4. When the job finishes, the Postgres and Redis containers are automatically destroyed.

5. Test the Production Build in E2E (Never the Dev Server)

Running your E2E tests against a development server (e.g., npm run dev with Vite, Next.js, or Webpack) creates a massive blind spot. Development servers inject hot-module reloading (HMR) scripts, skip asset minification, serve unbundled files, and bypass production caching layers.

If you test the dev server, you are explicitly testing code that your users will never see. Bugs caused by aggressive tree-shaking, CSS extraction failures, missing polyfills, or oversized production bundles will slip right past your tests and explode in production.

Instead of relying on external packages like start-server-and-test to manage this, modern tools like Playwright have native lifecycle management built in. You should configure your E2E test runner to strictly compile your code first, then boot a production-grade preview server for the entire stack (both backend and frontend) before running assertions.

The Anti-Pattern: Testing the Dev Server

The Playwright configuration blindly boots up the local development servers. It masks build errors, hides bundling issues, and runs significantly slower due to unoptimized assets and HMR overhead.

The Ugly Playwright Config:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  // BAD: Booting the dev server. If the production build fails, 
  // or the bundler breaks the CSS, this test will still pass.
  webServer: {
    command: 'npm run dev',
    url: 'http://127.0.0.1:5173',
    reuseExistingServer: !process.env.CI,
  }
});

The Production Standard: Native Multi-Server Lifecycle

Your CI pipeline (and local E2E runs) must replicate exactly what happens in production. By leveraging Playwright's webServer array, you can instruct the test runner to build and start both your backend API and your frontend static preview server. Playwright handles waiting for the URLs to respond, tearing down the processes when finished, and seamlessly reusing servers locally to save time.

The Clean Playwright Config:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  // GOOD: Playwright builds and serves the final production assets 
  // for BOTH the backend and frontend before running a single test.
  webServer: [
    {
      // 1. Compile and boot the production backend
      command: 'npm run build --prefix ../server && npm run start --prefix ../server',
      url: 'http://127.0.0.1:3000/graphql',
      reuseExistingServer: !process.env.CI,
      timeout: 300000,
      stdout: 'pipe',
      stderr: 'pipe',
    },
    {
      // 2. Compile and boot the production frontend preview
      command: 'npm run build --prefix ./ && npm run preview --prefix ./',
      url: 'http://127.0.0.1:4173',
      reuseExistingServer: !process.env.CI,
      timeout: 300000,
      stdout: 'pipe',
      stderr: 'pipe',
    }
  ]
});

By enforcing this, if a third-party dependency breaks your backend compilation, or a massive image fails to bundle properly in the frontend, Playwright will catch the crash exactly as a real user would experience it.

Closing

Testability is not a property you add to code after writing it. It is the result of specific design decisions made before the first function is ever written.

The architectural patterns across both of these parts cost almost nothing at design time, but they eliminate the majority of test infrastructure complexity downstream. It simply requires thinking about the boundaries of your code, controlling your state, and testing the exact systems your production environment will actually use. Stop fighting your test suite, and start designing for it.