product:test
Table of contents
- Introduction
- Running one level
- Testing several products
- The product’s own tests
- Handing variables to the own tests
- Production safety
- Reference
- See also
Introduction
product:test runs a product’s tests. It runs feature:test for every feature the product declares, then the product’s own tests under tests/. It works with or without a selected product.
splent product:test --product my_first_app
🧪 product:test — 1 product(s)
▶ my_first_app (7 features)
...
▶ my_first_app its own tests (1 file(s) in tests/)
...
tests/test_urls.py .... [100%]
============================== 4 passed in 1.02s ===============================
✅ All 1 product(s) passed.
Without --product, it tests the active product. product:release runs this command as one of its checks, and so does the test job of the product’s CI.
Running one level
splent product:test --unit
--unit, --integration and --functional run that level of the features’ tests only, as feature:test does. A level flag asks for a quick, partial run, so the product’s own tests wait for a run without one.
Testing several products
splent product:test --product app_a --product app_b --product app_c --unit
🧪 product:test — 3 product(s)
▶ app_a (5 features)
...
▶ app_b (3 features)
...
▶ app_c (7 features)
...
❌ 1/3 product(s) failed: app_b
Each product is tested even when an earlier one failed. A failure exits 1 and names the products that failed.
The product’s own tests
A product may keep tests of its own under tests/, in files named test_*.py or *_test.py. They check what no feature’s tests can, such as the URLs the product promises to keep serving.
They run in the product’s web container, on the test database, after the same safety checks as feature:test. They see the features the app loads, at the version the product pins or from the working copy an editable feature is linked to. The product’s src/ is on PYTHONPATH too.
They are not split by level. A run with a level flag says <product>'s own tests (tests/) run only without a level flag. and leaves them out. A product without tests/ runs its features’ tests alone, and one with nothing to collect passes.
Handing variables to the own tests
The tests get SPLENT_APP and what pytest reads, not the rest of your environment. When a test needs something only you have, such as a throwaway database server, name its variables in pyproject.toml.
[tool.splent.test]
env = ["MIGRATION_V2_TEST_*"]
export MIGRATION_V2_TEST_DB_HOST=migv2_db
splent product:test
handing the tests MIGRATION_V2_TEST_DB_HOST ([tool.splent.test] env)
An entry is a variable’s name or a prefix ending in *. The run names the variables it hands over, never their values. SPLENT_* and MARIADB_* are never handed, so a variable of yours cannot point the tests at another database. Nothing is started for you, and the features’ tests are not handed these variables.
Production safety
A product counts as in production when SPLENT_ENV=prod is in its docker/.env or in its running web container. One such product among those you test stops the whole run.
❌ Cannot run tests in production.
A production deployment is active for app_b.
Stop the deployment and start dev:
splent product:deploy --down
splent product:derive --dev
Reference
Options
| Option | Description |
|---|---|
--product NAME |
A product to test. Repeat it for several. Default, the active product |
--unit |
Unit tests only |
--integration |
Integration tests only |
--functional |
Functional tests only |
-v |
Verbose output |
Messages
| Message | Meaning |
|---|---|
ℹ️ <product>: no features declared. |
Only the own tests run, if any |
no tests collected in tests/ |
The product passes |
❌ Product not found: <product> |
No product of that name in the workspace |
❌ No product selected and no --product given. |
Select a product or pass --product |
[tool.splent.test] env names '<entry>', ... |
An entry is neither a variable’s name nor a prefix ending in * |
See also
feature:testtests the features of the active product, for day-to-day work.