Testing the CLI

How to write and run tests for splent_cli commands.

Table of contents

Overview
Running the tests
Test structure
The three testing patterns
Shared fixtures
What to assert
Testing error paths
Adding tests for a new command
Watch out for string substring traps

Overview

Testing a CLI is different from testing a library. The challenge is that commands do the following.

read environment variables (SPLENT_APP, WORKING_DIR)
touch the filesystem (pyproject.toml, cache, symlinks)
call external processes (docker, git, splent)

The solution is click.testing.CliRunner (Click’s built-in test runner) combined with pytest’s tmp_path fixture and unittest.mock.patch.

The key insight is that because all commands read context through the service layer (context.workspace(), context.require_app()), a single monkeypatch.setenv("WORKING_DIR", ...) redirects an entire command to a temporary directory. No Docker needed.

Running the tests

From inside the Docker container, run the tests as follows.

cd /workspace/splent_cli

# Run all tests
pytest tests/ -v

# Run a specific file
pytest tests/unit/commands/product/test_product_status.py -v

# Run a specific test class or case
pytest tests/unit/commands/env/test_env_list.py::TestFilter -v
pytest tests/unit/commands/env/test_env_list.py::TestFilter::test_filter_by_prefix -v

# With coverage
pytest tests/ -v --cov=splent_cli --cov-report=term-missing

Test structure

splent_cli/
  tests/
    conftest.py                          # shared fixtures (runner, workspace, product_workspace)
    unit/
      services/
        test_context.py                  # context.workspace(), require_app(), resolve_env()
        test_compose.py                  # project_name(), resolve_file(), normalize_feature_ref()
      commands/
        cache/
          test_cache_status.py
        env/
          test_env_list.py
        product/
          test_product_status.py
          test_product_up.py
        feature/
          test_feature_add.py

The three testing patterns

Every CLI command falls into one of three categories, each with its own testing approach.

Pattern 1. Pure logic (no mocking)

Commands that are just functions with no side effects (service helpers, string transformations).

from splent_cli.services.compose import project_name, normalize_feature_ref

def test_project_name_replaces_special_chars():
    assert project_name("splent_io/auth@v1.0", "prod") == "splent_io_auth_v1_0_prod"

def test_normalize_bare_name():
    assert normalize_feature_ref("splent_feature_auth") == "splent_io/splent_feature_auth"

Pattern 2. Filesystem commands

Commands that read or write files (pyproject.toml, .env, cache directories).

Use tmp_path + monkeypatch.setenv("WORKING_DIR", ...). No Docker needed.

from click.testing import CliRunner
from splent_cli.commands.cache.cache_status import cache_status

def test_cache_shows_versioned_feature(tmp_path, monkeypatch):
    monkeypatch.setenv("WORKING_DIR", str(tmp_path))

    # Build a fake cache entry
    cache = tmp_path / ".splent_cache" / "features" / "splent_io" / "splent_feature_auth@v1.0.0"
    cache.mkdir(parents=True)

    result = CliRunner().invoke(cache_status, [])

    assert result.exit_code == 0
    assert "splent_feature_auth" in result.output
    assert "v1.0.0" in result.output

Pattern 3. Subprocess commands

Commands that call docker, git, or other processes.

Mock subprocess.run to avoid needing Docker. Return a MagicMock with the right shape.

import json
from unittest.mock import patch, MagicMock
from click.testing import CliRunner
from splent_cli.commands.product.product_status import product_status

def test_shows_running_containers(product_workspace):
    containers = [{"Service": "web", "State": "running", "Publishers": []}]
    mock_output = json.dumps(containers[0])

    with patch("subprocess.run", return_value=MagicMock(returncode=0, stdout=mock_output, stderr="")):
        result = CliRunner().invoke(product_status, ["--dev"])

    assert result.exit_code == 0
    assert "web" in result.output
    assert "running" in result.output

Shared fixtures

Defined in tests/conftest.py. Import them by name; pytest injects them automatically.

`runner`

A CliRunner with stderr separated from stdout.

def test_something(runner):
    result = runner.invoke(my_command, ["--flag"])
    assert result.exit_code == 0

`workspace`

A tmp_path directory wired as the workspace root. Sets WORKING_DIR, clears SPLENT_APP and SPLENT_ENV.

def test_something(workspace):
    # workspace == Path("/tmp/pytest-.../test_something0")
    # WORKING_DIR is set to str(workspace)
    (workspace / ".env").write_text("FOO=bar\n")
    ...

`product_workspace`

A complete workspace with a test_app product. Sets WORKING_DIR, SPLENT_APP=test_app, SPLENT_ENV=dev. Creates pyproject.toml and docker-compose files.

def test_something(product_workspace):
    # Ready to test any product:* command
    with patch("subprocess.run", ...):
        result = runner.invoke(product_up, ["--dev"])
    assert result.exit_code == 0

Helper functions

from tests.conftest import make_env_file, make_cache_entry

# Write a .env file into a workspace
make_env_file(workspace, "SPLENT_APP=my_app\nGITHUB_TOKEN=secret\n")

# Create a cache directory entry
make_cache_entry(workspace, "splent_io", "splent_feature_auth", "v1.0.0")  # versioned
make_cache_entry(workspace, "splent_io", "splent_feature_notes")           # editable

What to assert

result from runner.invoke() has three useful attributes.

Attribute	What it contains
`result.exit_code`	`0` = success, `1` = error, `0` with `raise SystemExit(0)` = cancelled
`result.output`	Everything printed to stdout (and stderr if `mix_stderr=True`)
`result.exception`	The exception if the command crashed (not `SystemExit`)

Always assert exit_code first. If a test fails unexpectedly, print result.output to debug.

result = runner.invoke(my_command, ["--flag"])
print(result.output)          # see what was printed
print(result.exception)       # see if it crashed
assert result.exit_code == 0

Testing error paths

Error paths are as important as the happy path. For every command, test the following.

# Missing required env var
def test_requires_splent_app(runner, workspace):
    result = runner.invoke(product_up, ["--dev"])
    assert result.exit_code == 1
    assert "SPLENT_APP" in result.output

# Mutually exclusive flags
def test_rejects_both_dev_and_prod(runner, product_workspace):
    result = runner.invoke(product_status, ["--dev", "--prod"])
    assert result.exit_code == 1
    assert "Cannot specify both" in result.output

# Missing file
def test_exits_when_no_env_file(runner, workspace):
    result = runner.invoke(env_list, [])
    assert result.exit_code == 1
    assert ".env" in result.output

Adding tests for a new command

Create tests/unit/commands/<group>/test_<command>.py
Import the command function directly (not through the CLI entry point)
Use the appropriate fixture (workspace, product_workspace, or tmp_path)
Mock subprocess if the command calls external tools
Test, at minimum, flag validation, the happy path, and one error path

from click.testing import CliRunner
from splent_cli.commands.mygroup.my_command import my_command

class TestFlagValidation:
    def test_requires_splent_app(self, runner, workspace):
        result = runner.invoke(my_command, [])
        assert result.exit_code == 1
        assert "SPLENT_APP" in result.output

class TestHappyPath:
    def test_success(self, runner, product_workspace):
        result = runner.invoke(my_command, ["--some-flag"])
        assert result.exit_code == 0
        assert "expected output" in result.output

Watch out for string substring traps

When asserting that a variable name does NOT appear in output, make sure your test variable names are not substrings of each other. For example, "SET_VAR" is a substring of "UNSET_VAR", so assert "SET_VAR" not in output would fail even when SET_VAR itself is absent.

Use clearly distinct names like LOADED_KEY and MISSING_KEY.