Otoroshi Challenge

Putting an Otoroshi gateway in front of an application moves a whole class of concerns out of your code. Authentication through OAuth2, OpenID Connect, JWT verification or API keys, rate limiting and quotas, request and response transformation, canary releases, and a Coraza web application firewall all become route configuration, applied the same way to a website or to an API. Your teams change these behaviours from the gateway, without a deployment and without touching the applications behind it.

That model only holds if those applications cannot be reached any other way. An application deployed on Clever Cloud keeps its own public URL, and anyone who finds it bypasses the gateway along with everything it enforces.

The Otoroshi challenge closes that gap without a private network. Request Flow runs a middleware in front of your application that answers 401 Unauthorized to every request unable to prove it came through your gateway, apart from the paths you exclude from the challenge, and forwards the rest untouched. Your code changes in no way, and your application never sees the challenge headers.

A Network Group reaches the same result differently, by placing your Otoroshi add-on and your applications on a private WireGuard network and keeping those applications off the public internet. The challenge suits applications that stay on a public URL, or that you want to protect without changing their network topology. Nothing prevents combining both.

The mechanism is version 2 of the Otoroshi communication protocol, a signed token exchange based on a secret shared between the gateway and the middleware.

Enable the Otoroshi challenge

Set the shared secret, and Request Flow detects and starts the middleware on its own:

OTOROSHI_CHALLENGE_SECRET="a-secret-shared-with-your-gateway"

Use the same value in the secure communication settings of the Otoroshi route pointing at your application. Generate one with openssl rand -base64 32, and use a different secret for each application.

Automatic detection is enough in most cases. Set CC_REQUEST_FLOW explicitly when you chain several middleware services, listing them in the order you want from the public port to your application:

CC_REQUEST_FLOW="otoroshi-challenge,varnish"

How the exchange works

Your gateway sends a JWT in an Otoroshi-State header on every request. The middleware verifies it, and answers with a second JWT in an Otoroshi-State-Resp header that proves it holds the same secret. A response without that header tells the gateway the backend is not the one it expects.

The incoming token is signed with HS512 and must carry three claims. The iss claim has to be Otoroshi exactly, state holds the value the middleware echoes back, and exp has to be in the future. The iat and nbf claims are accepted but optional.

The middleware answers with a token signed the same way, carrying the received state value in a state-resp claim, an aud claim set to Otoroshi, and a 30 second validity:

{
  "state-resp": "the value received in the state claim",
  "aud": "Otoroshi",
  "iat": 1788585098,
  "nbf": 1788585098,
  "exp": 1788585128
}

The Otoroshi-State header stops at the middleware and never reaches your application, so nothing downstream has to know about the protocol.

Listen on the right port

The middleware takes the public port 8080 and forwards to your application on port 9000. When your application manages its own HTTP server, it has to follow:

PORT="9000"

Clever Cloud configures the backend itself on runtimes with a managed web server or port. In every other runtime, an application still bound to 8080 fails to start, because the middleware already holds that port:

Error: listen EADDRINUSE: address already in use 0.0.0.0:8080

Keep the health check working

The platform health check requests your application through the public port, where the middleware now answers 401 to a request carrying no challenge. The default check accepts any status from 200 to 499, so it keeps passing.

Setting CC_HEALTH_CHECK_PATH narrows that range to 200 to 299. The middleware reads it, along with CC_HEALTH_CHECK_PATH_0, CC_HEALTH_CHECK_PATH_1 and the following ones up to the first missing index, and lets GET and HEAD requests on these paths reach your application with no challenge:

CC_HEALTH_CHECK_PATH="/health"

Exclude another path the same way with OTOROSHI_CHALLENGE_EXCLUDE_PATHS, a comma-separated list merged with the health check paths:

OTOROSHI_CHALLENGE_EXCLUDE_PATHS="/health,/ready"

Paths are compared exactly and case-sensitively, ignoring the query string and any trailing slashes. /health therefore covers /health/ and /health?verbose=true, but neither /Health nor /health/database. A request carrying an Otoroshi-State header goes through the usual verification even on an excluded path, so your gateway keeps calling these endpoints as it does the rest of your application. An excluded path answers every other GET and HEAD request from the public internet without any verification, so keep the list to endpoints returning a status and nothing else.

Troubleshooting

The middleware answers rejected requests with a JSON body naming the reason, which is the fastest way to tell a configuration mistake from a gateway one:

A Some software are not listening as expected: 9000 error in the deployment logs means the middleware runs but your application does not listen on 9000. Read Listen on the right port.

Confirm the protection from your machine, where the challenge is missing by design:

curl -s -o /dev/null -w "%{http_code}\n" https://your-application.cleverapps.io/

A 401 answer means only your gateway can reach the application. A 200 means the middleware is not running, so check that OTOROSHI_CHALLENGE_SECRET is set and that your runtime supports Request Flow.

• Learn more about Request Flow
• Deploy an Otoroshi add-on
• Learn more about the Otoroshi communication protocol