We recently reworked how they were handled. The rework brings clarity and consistency to configuration variables, with some breaking changes. In this article, we detail what changed and what we did to prevent you from encountering the errors.
Configuring your deployment
A bit of context
The Clever Cloud platform tries to build and run your applications by detecting the technology you are using: a composer.json file exists? We run composer install. You already have a composer.phar in your repo? We use this one instead of the system one.
However, you sometimes have to tell us what to do. Your application needs a specific version of PHP or Java to work? We won’t magically know it. That’s where configuration variables kick in: to make us know which PHP version to use or whether we need to enable some PHP extension, you need to use Environment Variables that will be interpreted by the software that manages the application deployment.
An empirical configuration handling
Historically, we started using environment variables to change the behaviour of the deployment process. The process itself would check for the presence and validate the variables when it needed to use it. The support for each variable was empirical and subject to variations.
For example, to enable the APCu PHP extension, you can set ENABLE_APCU="true". Well, in the extension enabler script, the value of the ENABLE_APCU variable was checked just before activating (or not) that extension. This particular piece of code would accept the string true in a case-insensitive way.
Somewhere else in the code, CC_CACHE_DEPENDENCIES would be only checked for the exact string “true”. So “True” or “TRUE” would be considered as false-y values. For some other variables, “yes” and “no” where allowed instead of or in addition to “true” and “false”.
Different variables meant different validation strategies. It made customer support harder, as one would have to take a look at the actual code to help confused customers.
This software was long due for a do-over.
Clean early, validate early, abort early
These past few months, we started rewriting how the 300+ environment variables are handled. It needed to:
- Locate all the variables used
- List all the ways they were used
- Document the different types of variables that exist
- Clean everything up
LLMs to the rescue!
As written before, this cleanup task was long overdue. It was actually in the minds of a few of us for some years already. However, the day-to-day work, the new features to produce, the rapid growth of the company, etc. were all getting in the way of this huge rewrite.
The longer it went, the more variables we added and the bigger the work was. That’s where LLMs arrived. Finding patterns and rearrange text is their strong suit! Armed with these tools, tackling on this huge work suddenly seemed possible.
Centralizing everything
We started by listing all the variables. With the help of LLMs, all the variables were found and centralized in a single file. This file now acts as the single source of truth for all the variables. We also added types to the variables and noted how each variable was used.
In the future, this single file will be used to generate documentation and validation in early our APIs and clients.
Types, the foundation stone for healthy code
When constructing programs that take data from customers, you want to uphold these two principles:
- Customers are chaotic: they expect to be understood whatever they send.
To provide a seamless experience, you want to accept a wide range of values from the customers.
If you don’t accept the values, print user-friendly messages on what is wrong. - Inside of your program, you want things to be consistent, simple to use, and predictable.
To achieve that, one of the best ways is to use strong type inside the program. For example, take the “true” string: it indicates that we are handling a boolean value, true or false. In the program, we only check that the value is either “true” or “false” (any other values meaning false). Let’s use a boolean, then.
Throughout the whole deployer program, boolean values are exposed to the customers in various ways:
- “true” / “false”
- “yes” / “no”
- “enable” / “disable”
- “enabled” / “disabled”
- “1” / “0”
- … (yes, there are more!)
Then why not accepting all of them? That’s what we decided.
For boolean values, customers can now use any of these values:
- For true: 1, true, yes, on, enable, enabled
- For false: 0, false, no, off, disable, disabled
… plus some specific true-y or false-y values for backward compatibility reason.
Then in the code, we fetch the variable and we get a boolean value. No more parsing the same strings in dozens of different functions!
In the end, we came up with these types:
- Boolean
- Disabler (almost a boolean, but “disable” means true)
- Alternatives (choosing in a given list, like java versions)
- Number
- String
For each type, we perform the same sanitizing steps. We raise the consistent errors.
Abort early
In the previous version of the deployment code, variables values where checked at the moment of usage. This meant that a variable used to start your application was validated after building your applications! You’d have to wait for your whole build to know if your variable was correct. Then you’d fix your variable and retry. Then you’d get another error on another configuration variables, used later in the process. 🤦
To respect the two principles stated above, we had to sanitize everything at the start.
The deployment program now checks all your variables at the beginning. This means that no matter how many bad variables you set, you get a list of errors at the beginning of the deployement, which will stop there. No more waiting 20 minutes before getting an error!
Consistency makes for stricter behaviour
Since each part of the code was handling the parsing of customer-provided string values, different behaviours appeared:
- Sometimes, an unexpected value would raise an exception
- Sometimes, it would just be ignored
- Sometimes, it would be ignored and then made customer’s applications fail for obscure
reasons
This new validation being consistent, it became stricter that before for some variables. We also chose to consider all unsupported values as a failure. This means that your application that deployed fine might suddenly fail to deploy because the value of a variable is not supported anymore.
Clever Cloud blue-green deployments will prevent downtime in case of a bad variable value! Deployment will fail early and your already running instances will continue to serve the application.
Before releasing this update, we also went over all your Clever Cloud configuration variables. We tested them and tried to fix them before rolling out the update.
What does it look like?
If your application deployment fails because environment variables not passing validation, you will see this message in the logs:
This states that there are four variables that have an unexpected value. The proper values are given in the error message.
When you get this message, go to the “Environment variables” tab of your app in the console and fix the variables, e.g.:
CC_CGI_IMPLEMENTATION: there was a typo, easy to fix!CC_COMPOSER_VERSION: the variable is defined, but the value is an empty string.
If you fix all these errors, that’s it! No new unsupported variable error will arise during the next deployment.