Features that run a server

Some features need more than Python. A search feature needs a search node, a session feature needs Redis, a reverse proxy feature needs nginx. A feature says so by shipping a top-level docker/ directory, and from then on every product that installs it gets its own instance of that stack.

Table of contents

• The rule
• What a feature ships
  • 1. The service is named after the package
  • 2. No container_name
  • 3. Every published port comes from a *_HOST_PORT variable
  • 4. Data goes in a named volume
• What the CLI does
  • A project name per product
  • A network per product
  • The env file it reads
• Checking it
• Migrating a feature that predates the rule

---

The rule

One product, one instance. Two products of the same line must be able to run at the same time, on the same machine, without knowing about each other.

That is not a nicety. A product line exists to produce several products from one set of features, so two of them running side by side is the ordinary case, not an exotic one. Everything below is what it takes for that to hold.

The CLI does its half automatically. What a feature has to do is keep four rules in its compose files, and check:infra reports every one of them.

---

What a feature ships

splent_feature_elasticsearch/
└── docker/
    ├── docker-compose.dev.yml     the stack in development
    ├── docker-compose.prod.yml    the stack in production
    └── .env.example               its defaults, per variable

splent feature:create <org>/<name> --docker scaffolds all three with the rules already applied and every value to fill in marked CHANGEME.

1. The service is named after the package

services:
  splent_feature_elasticsearch:      # not "elasticsearch", not "es"

On the network this name is the hostname other containers reach the service by, and it is what the feature’s own configuration points at (ELASTICSEARCH_URL=http://splent_feature_elasticsearch:9200). Shortening it means every product has to know the short name too.

2. No container_name

Compose prefixes the names it derives with the project name, which is per product. It does not prefix a container_name you write yourself, so the second product to start collides with the first and Docker refuses.

3. Every published port comes from a *_HOST_PORT variable

    ports:
      - "${ELASTICSEARCH_HOST_PORT}:9200"

product:env --merge gives each product a different value for it, offsetting the feature’s default by zlib.crc32(<product>) % 1000. See Port offset. A literal number defeats that and the second product dies with port is already allocated.

The container side stays fixed: nothing outside the network reaches it.

The variable must have a default in docker/.env.example. Compose does not treat an unset variable in a port mapping as an error; it publishes on a random host port instead. The stack comes up, reports success, and listens where nobody can predict, which is the exact failure the offset exists to prevent, arrived at silently.

4. Data goes in a named volume

    volumes:
      - elasticsearch_data:/usr/share/elasticsearch/data
volumes:
  elasticsearch_data:

Compose prefixes a named volume with the project name, so each product keeps its own data. A host path is not prefixed, and every product would write into the same directory. Mounting a host path read-only is fine, and is how a feature ships configuration files.

---

What the CLI does

A project name per product

product:up runs each feature’s stack as a Compose project named after the product, the feature and the pinned version:

egc_wiki_splent_io_splent_feature_elasticsearch_v0_1_0_dev

The version is in there so a product moving to a new feature version gets a new stack rather than reusing the old one. product:down, product:restart, product:clean, product:containers and feature:install all derive the same name, so no command can address a stack another one started.

In production there is no separate stack: product:build merges every feature’s compose fragment into the product’s own docker-compose.deploy.yml, and product:deploy runs that under <product>_deploy.

A network per product

A project name of its own gives each product its own containers. It does not give them their own names. Compose puts every container on the network under a DNS alias equal to its service name, so on one shared network two products running the same feature publish the same alias, and Docker’s resolver answers with both addresses in arbitrary order.

Measured on two wikis of one line, from one product’s web container:

$ for i in 1 2 3 4 5 6; do getent ahostsv4 splent_feature_elasticsearch; done
172.18.0.4   ← this product's node
172.18.0.5   ← the other product's node
172.18.0.4
172.18.0.4
172.18.0.4
172.18.0.5

Two lookups in six went to the wrong node, which has none of this product’s indices. So each product gets a network of its own, named after it:

networks:
  splent_network:
    external: true
    name: ${SPLENT_NETWORK:-splent_network}

The key stays splent_network, so nothing that references it changes. What changes is the network it resolves to. product:env --merge writes SPLENT_NETWORK=<product>_network into the product’s merged env, and product:up creates the network before the first container starts. The fallback keeps a product written before this rule working unchanged.

It also ends an access nobody intended: every product’s database used to sit on the shared network under a name any other product’s code could resolve.

The env file it reads

A feature’s compose file sits in the feature’s directory, so Compose would read the feature’s .env and never see the product’s values. Every command therefore passes --env-file explicitly, pointing at the merged file for the environment in play: docker/.env in development, docker/.env.deploy in production.

---

Checking it

splent check:infra

  Isolation
  [OK] 1 feature stack(s) isolate per product

Every rule above is reported here, against the compose file as written: docker compose config resolves ${ELASTICSEARCH_HOST_PORT} to a number, which is indistinguishable from a hardcoded one, so the check that matters most can only be made before substitution.

None of these break a single product, which is why they need a check rather than a test run. They break the first time two products of one line come up together, and by then the symptom is a random port or an intermittently wrong answer rather than an error.

---

Migrating a feature that predates the rule

1. Remove any container_name.
2. Replace literal host ports with ${<FEATURE>_HOST_PORT} and add the default to docker/.env.example.
3. Add name: ${SPLENT_NETWORK:-splent_network} to the external network.
4. Move writable host paths to named volumes.
5. splent check:infra until the Isolation section is clean.

The first product:up afterwards creates a new, empty volume, because the volume name now carries the product’s name. The old shared volume is left untouched; nothing deletes it. For a feature holding a cache or an index, such as elasticsearch, rebuild it rather than migrate it:

splent search reindex

Containers left running under the old shared project keep the ports and the service alias the new stack wants. product:up, product:down and product:containers report them with the command that removes them.