English
Deutsch
PolskiTraefik Labels in Detail, Including the Dollar Trap
Contents
Traefik is configured by writing labels on a container, which sounds like a list of settings and is really a namespace. The keys are paths through a tree, and what looks like six independent lines is three objects that know about each other through their names – except the service, which Traefik attaches on its own to a router without service= as long as the container declares exactly one.
That construction is what makes the configuration short and what makes its failures quiet. A typo in a key produces a second object rather than a complaint, and a name is only checked where a router references it – with a log entry and a 404 as the only outward signs.
Three Objects, One Name
A router decides which requests belong to this container, a service says where to send them, and a middleware modifies them on the way. Each lives under its own branch of traefik.http, and each is identified by a name that is chosen freely – the label key is the declaration.
labels:
- "traefik.enable=true"
- "traefik.http.routers.blog.rule=Host(`blog.example.com`)"
- "traefik.http.routers.blog.entrypoints=websecure"
- "traefik.http.routers.blog.tls.certresolver=le"
- "traefik.http.routers.blog.middlewares=blog-auth@docker"
- "traefik.http.middlewares.blog-auth.basicauth.users=admin:$$2y$$05$$e6zX…"
- "traefik.http.services.blog.loadbalancer.server.port=8080"No line says that the router and the service belong together, and it is not the shared name blog that binds them. When a router names no service, Traefik attaches the only service the container declares; when the container declares several, Traefik drops the router with a log entry. A misspelled traefik.http.services.blogg.loadbalancer.server.port therefore has no effect in this example. It becomes serious once the router names its service explicitly: with traefik.http.routers.blog.service=blog it points at a service that does not exist; Traefik marks the router as failed with “the service … does not exist” and answers requests for that host with a 404.
When the container exposes exactly one port, the service label can be left out entirely and Traefik infers it. That convenience is worth knowing about mostly so it can be recognised in someone else’s file – as soon as a second port appears, Traefik takes the lowest one, and the route breaks at the moment an unrelated, lower port is added.
The Provider Suffix
Middlewares are referenced by name, and Traefik keeps a separate namespace per provider – the Docker labels, a configuration file, a Kubernetes resource. A bare blog-auth is looked up in the router’s own provider, with no fallback to other providers. For a middleware from the same labels it is therefore enough, and blog-auth@docker merely spells out what applies anyway; the suffix is only needed for a reference across the provider boundary, such as blog-auth@file for a middleware from Traefik’s configuration file.
A wrong reference – misspelled, or without a suffix to a middleware from another provider – does not open the site without authentication. Traefik marks the router as failed with “middleware … does not exist”, does not set it up and answers requests for that host with a 404. The only quiet case is a reference that is missing altogether: a router that does not list the middleware serves the application without the authentication that was supposed to sit in front of it, and nothing in the log mentions it, because from Traefik’s point of view none was requested.
The Dollar Trap
A bcrypt hash looks like $2y$05$e6zX... and consists largely of dollar signs. Docker Compose performs variable substitution on every value it reads, so $2y becomes an empty string, $05 becomes an empty string, and the password that reaches Traefik is a fragment that matches nothing.
# generate
htpasswd -nbB admin "passwort"
admin:$2y$05$e6zXx8Q0m2yYy1H4o1Yb1uJf2Rr7cKq5dW6vT8sN0pL3aG9hC1dS2
# in docker-compose.yml every $ is doubled
admin:$$2y$$05$$e6zXx8Q0m2yYy1H4o1Yb1uJf2Rr7cKq5dW6vT8sN0pL3aG9hC1dS2
# in a plain labels file or on the command line it stays singleThe doubling is a Compose rule, not a Traefik rule, which is why the same string has to be written differently depending on where it lives. A hash placed in Traefik’s own dynamic configuration file keeps its single dollars; the same hash in a compose file needs them doubled; and moving a working configuration from one to the other breaks it in exactly this way.
The symptom is a login prompt that refuses every password, including the right one, which sends people to check the password rather than the file that ate it.
What Each Mistake Looks Like from Outside
| Mistake | Symptom | Where to look |
|---|---|---|
| Router names a service that does not exist | 404 for that host | the dashboard: the router with the error “the service … does not exist” |
| Middleware reference misspelled or without a suffix to another provider | 404 for that host | the dashboard: the router with the error “middleware … does not exist” |
| Single dollar signs in compose | 401 for every password | docker inspect: the label is shorter than the hash |
| Container not on the proxy network | 502 or 504 | the service has no reachable server |
traefik.enable=true missing, with exposedByDefault=false |
404, nothing appears at all | the dashboard: the container is not listed |
| Two containers, same router name, different settings | 404, Traefik drops the router | the log: “defined multiple times with different configurations” |
Four of the six are answered by the dashboard, which is the argument for enabling it on an internal entrypoint even on a small machine. It shows what Traefik actually understood, and every one of these mistakes is a difference between what was written and what was understood.
Two Habits That Prevent Most of It
The first is to derive every name from the service and use it unchanged in all three places – container blog, router blog, service blog, middleware blog-auth. A name that is mechanical cannot be misremembered, and a diff between two containers’ labels then shows only the lines that genuinely differ.
The second is to read the labels back rather than the file. docker inspect shows the values as the daemon stored them, after Compose has finished substituting – which is where a doubled dollar becomes a single one again, and where a hash that survived is visibly the same length as the one that was generated.
Questions and answers
Can the hash be stored somewhere Compose never gets to see it?
Yes, in two ways, and both keep the dollar signs single. The first is a users file: instead of basicauth.users, the label basicauth.usersfile takes a path, and the file holds the lines exactly as htpasswd prints them. Compose only reads the path, not the contents, so there is nothing for it to substitute. The file is opened by Traefik, though, not by the application, so it has to be mounted into the Traefik container rather than into the container it protects.
The second way moves the whole middleware into Traefik’s own dynamic configuration file, where the hash is written with single dollars anyway. That changes its provider, however, and this is where the suffix from the second section comes in: the router then has to reference blog-auth@file instead of blog-auth@docker. A reference left unchanged during the move points at a name that no longer exists in that provider, and Traefik takes the router out of service with “middleware … does not exist” until the reference is correct. A look at the dashboard, to confirm that the router shows no error, is therefore part of the move.
Does a middleware apply to every route that leads to the same service?
No. A middleware is attached to a router, not to the service, and only acts on the requests that router accepts. A second router pointing at the same service, for a further host name or through a different entrypoint, serves the application without authentication unless the middleware is in its list as well; the dashboard shows this for each router separately.
Why does a container outside the proxy network produce a 502 or 504 rather than a 404?
Because in that case everything that was written is correct. The router and the service exist and match, so the request is accepted and passed on. It only fails at that point: Traefik has an address for the container from Docker but cannot reach it, because the two share no network. A 404 arises when Traefik finds no valid target for a request; a 502 or 504 arises when it has found a target that does not answer.
More common than a missing network is a container attached to several. Traefik then uses one of its addresses, and without an explicit setting there is no telling which. The label traefik.docker.network pins the network. It expects the name under which Docker knows the network, and Compose usually prefixes networks it creates with the project name. As with the dollar signs, docker inspect shows which name applies.
Sources
Related articles
