tl;dr: On March 21th 2022 the Cellar C1 cluster will reach its end of life. So we ask you to perform a migration to a newer cluster Cellar, which will also improve performance. The C1 cluster did not accept new add-ons since the 20th January 2019.

In fact, data need to be migrated from this old add-on to a new Cellar add-on. Newly created Cellar C2 add-ons come with various improvements :

• A better S3 protocol support (including the AWS V4 signature)
• Improved upload and download performances (x3 for upload, x2 for download)
• Improved resilience and availability of your data (Data is now replicated across two datacenters more than 20km apart)

How to migrate my data?

We can help you to migrate your data by syncing it directly on our side. To do that, reach the support and include in your message source and destination add-ons, including the buckets to migrate. The support team is here to help.

If you wish to migrate on your side, or at least keep your buckets synchronized, we built a simple tool to help you: Cellar migration tool.
You can run this tool as many times as you need. It will only synchronize objects that are different between the source and the destination buckets.
If you do not feel comfortable using it, you can also use the s3cmd sync command from the s3cmd tool.

If your application handle it, we encourage you to upload new objects as soon as possible to the new bucket. Then when you need to query that object, your application can query both buckets (old and new) to get it. You can also store the complete URL to make this easier. This should help you transition from one bucket to another without downtime.

Please note that we can't make sure that the buckets names you use will be available on the destination cluster. If you have a name conflict, please contact us and we will work something out.

A few days before the deadline, we will have short service brown-outs to remind customers that the service is going to be shut down. Those brown-outs will be announced on our status page a few days before. Make sure to subscribe to Status Updates to receive a notification.

What will happen if I can't do the migration?

If you can't do the migration or have any questions about it, we strongly recommend that you contact our support team via the console.
Before shutting down the cluster, we will backup your data and keep it for 6 months. Our support team will be able to provide you a download link or re-inject the data in a new bucket if you wish to recover it.

Clever Cloud applications to run your jobs

Enhance your Clever Cloud deployments workflow

pipeline {
  environment {
    CLEVER_TOKEN = credentials('CLEVER_TOKEN')
    CLEVER_SECRET = credentials('CLEVER_SECRET')
  }

  stage('clever-tools') {
      steps {
        script {
          sh 'npm install -g clever-tools'
       }
     }
  }

  stage('tests') {
    // Your tests steps
  }

  stage('deployment') {
    steps {
      script {
        // Deploy the application
        // A .clever.json file must exists in the repository
        sh 'clever deploy'
      }
    }
  }
}

Built with the classic features you love

Pricing

What's next?

A connection pool will cache your database connections and keep them open for future requests

Configure ProxySQL

• CC_MYSQL_PROXYSQL_USE_TLS: A boolean (true or false) to enable TLS between ProxySQL and your MySQL add-on. Defaults to true.
• CC_MYSQL_PROXYSQL_MAX_CONNECTIONS: An integer which defines how many maximum connections ProxySQL will open to your MySQL add-on. Defaults to 10. You can refer to our ProxySQL documentation to learn more about how you should compute the maximum connection value when auto scalability is involved.

Examples

PHP using PDO

<?php
// This variable is injected during the deployment
$socket = getenv("CC_MYSQL_PROXYSQL_SOCKET_PATH");
// Get the database name from the environment
$db = getenv("MYSQL_ADDON_DB");
// Get the database user from the environment
$user = getenv("MYSQL_ADDON_USER");
// Get the database password from the environment
$pass = getenv("MYSQL_ADDON_PASSWORD");
$dsn = "mysql:unix_socket=$socket;dbname=$db";
try {
  $pdo = new PDO($dsn, $user, $pass);
} catch (PDOException $e) {
  throw new PDOException($e->getMessage(), (int)$e->getCode());
}

Wordpress

// To connect using a unix socket, the syntax is: `localhost:/path/to/socket`
define('DB_HOST', "localhost:" . getenv("CC_MYSQL_PROXYSQL_SOCKET_PATH") );
// Get the database user from the environment
define('DB_USER', getenv("MYSQL_ADDON_USER"));
// Get the database password from the environment
define('DB_PASSWORD', getenv("MYSQL_ADDON_PASSWORD"));
// Get the database name from the environment
define('DB_NAME', getenv("MYSQL_ADDON_DB"));

Node.js

const mysql      = require('mysql');
const connection = mysql.createConnection({
  // Get ProxySQL unix domain socket path from the environment
  socketPath : process.env["CC_MYSQL_PROXYSQL_SOCKET_PATH"],
  // Get the database user from the environment
  user       : process.env["MYSQL_ADDON_USER"],
  // Get the database password from the environment
  password   : process.env["MYSQL_ADDON_PASSWORD"],
  // Get the database name from the environment
  database   : process.env["MYSQL_ADDON_DB"]
});
connection.connect(function(err) {
  if (err) {
    console.error('error connecting: ' + err.stack);
    return;
  }

  console.log('connected as id ' + connection.threadId);
});

Metrics

Documentation

Conclusion

Parse Server

Deploy on Clever Cloud

1. First, you need to clone this repository (or use an existing one) and modify the Parse Server instanciation:

   var api = new ParseServer({
     databaseURI: process.env.MONGODB_ADDON_URI, // Use the MongoDB URI
     cloud: process.env.CLOUD_CODE_MAIN || __dirname + '/cloud/main.js',
     appId: process.env.PARSE_APPID, // Use environment variable to set the APP_ID
     masterKey: process.env.PARSE_MASTERKEY // Use environment variable to set the PARSE_MASTERKEY
   });
   

2. Open the Clever Cloud Console, create an account if you don't have one yet
3. Create a Node.js application and a MongoDB addon
4. Add these two environment variables: PARSE_APPID: <parseAppID>, PARSE_MASTERKEY: <parseMasterKey>
5. Follow the instructions about adding the remote to your local git repository
6. Deploy it: git push clever master and head to your application's logs

You may have noticed major updates on the Clever Cloud Console these last weeks. Along with the quick search feature, keyboard shortcuts and a new UI, we introduced the use of service workers to reduce the load time of the Console.

Service workers you said? Surely you mean Web Workers, right?

No! Service workers are scripts which are run by your browser, in the background. They are completely separated from the web pages of your site and have multiple uses. You can use them for HTTPS proxying, push notifications, or even background syncing.

Today, we will cover HTTPS proxying.

The Console should load faster

When you load the Clever Cloud Console, we need to fetch data (the user, its organisations / apps / addons…) to bootstrap the whole application. If you have a lot of applications / belong to a lot of organisations, this may take a while, up to several seconds ("a lot" can mean ~200 apps for some of ourusers). This data isn't updated a lot. Most of our customers are using the Console to look at the logs or configure their applications. So it needs to be fast for them to access / set the information they want.

Here come the service workers. They can cache the response of any HTTPS request that is emitted from your application (which can be used to build offline apps). What we did is that any huge XHR request done when the Console is loading is now served from the cache if it exists.

How does it work?

• These new APIs (Service Workers, Cache API, Fetch API) are heavily using ES6 Promises, you should be familiar with them
• Incidentally, you can use ES6 because browsers that support service-workers are also implementing mainstream ES6 features
• You need to use a recent browser: Chrome 42+, Firefox 44+
• Service Workers are HTTPS only, they won't work on HTTP requests. Chrome will allow http://localhost as a secure origin though.

How to debug

• You can check "Enable Service Workers over HTTP (when toolbox is open)" on Firefox Devtools to use service workers on any HTTP request.
• Chrome will only allow localhost or HTTPS urls. Good thing that HTTPS is available on *.cleverapps.io.
• Firefox and Chrome have an endpoint to list service workers: Firefox: about:serviceworkers, Chrome: chrome://inspect/#service-workers
• You can use regular console.log() to log from the service worker.
• You won't be able to see intercepted requests in the Network panel of Firefox (as of Firefox 45.0a1).

The code

First, we need to setup our service worker and ask the browser to register it (in your existing javascript application):

  navigator
    .serviceWorker
    .register('/service-worker.js')
    .then(navigator.serviceWorker.ready)
    .then(registration => {
      console.log('Server worker has been registered');
    });

You don't have to worry about multiple registrations: if a service worker has already been registered, the browser will simply ignore it.

Once this is done, we have to create a service-worker.js file. Service Workers are event based. Two events will be useful in our case: install and fetch. install will allow us to initialize our service worker (like pre-caching assets we haven't already loaded). fetch will be triggered each time a request is intercepted for the current scope.

We will use the Cache API to cache requests / responses.

'use strict';

const CACHE_URLS = [
  "/img/img1.png",
  "/img/img2.png",
  "/users/me"
];

const CACHE_NAME = "cc-cache-test";

self.addEventListener('install', event => {
  event.waitUntil(
    caches
      .open(CACHE_NAME)
      .then(cache =>{
        cache.addAll(CACHE_URLS);
      })
      .catch(e => console.error("Couldn't install the service worker, reason:", e));
  );
});

self.addEventListener("fetch", event => {});

Here, during the installation step, the service worker will download and cache the CACHE_URLS urls even before the user requests them. Now, we need to tell our SW to use the cache whenever we have a cached version of the request or to fetch it if we don't. We can use the great HTML5 Fetch API:

self.addEventListener("fetch", event => {
  event.respondWith(
    caches.open(CACHE_NAME)
      .then(cache => cache.match(event.request))
      .then(response => {
        if(response){
          return Promise.resolve(response);
        } else{
          return fetch(event.request).then(res => {
            if(res && res.status === 200){
              caches.open(CACHE_NAME)
                .then(cache => {
                  cache.put(event.request, res);
                });
            }
            return res.clone();
          });
        }
      })
  );
});

What we are doing is to open the cache and then try to match the current request. If we have a match, we return the response associated to the request. If we don't, we fetch the request, check its response code and save it into the cache if it succeeded. Next time, it will load faster!

Note that we have to use res.clone();. This is because the response is a stream which can be read only once. Here, we want to store its data into the cache AND return the same data to the browser. This won't work unless we .clone() it first.

Now, our service worker is ready to be used.

Sounds perfect but…

A major drawback is that you need a quite recent browser and these API may differ between browsers. At the time of writing, this won't work on Safari (any version), Firefox stable (aurora/dev has some support), Opera, Internet Explorer (Edge does support some features). You will have to handle these incompatibilities yourself.

But, even if service workers are not yet supported by all browsers, your application will still be able to run in a normal way. Only proxying and cache features will be affected, requests will be done as they were before. This is completely backwards compatible.

How do we use it at Clever Cloud

We can't store all of the data or cache invalidation will be a mess to handle. Instead, we only store a few defined requests. When we load the Console, we use the cache, then wipe and update it in the background. The user can use the Console with quite fresh data and the UI will be updated as soon as we have the new one.

This feature allowed us to speed up the initial load of the Console, going well under 1 second if the cache exists. In a near future, we could also use service workers with the Notifications API to display application deployments's state even when the Console is not open in your browser.

More reading

To learn about service workers, I used a few links:

• Service Workers
• Cache API
• ES6 Promises
• HTML5 rocks (may be a little out of date but still accurate and well explained)

Lately, a lot of developpers asked us about Meteorjs support on Clever Cloud. This is indeed a growing technology out there, so I decided to test it myself.

What is Meteorjs ?

Meteor.js is based on Nodejs and is a real-time cross-platform web application framework (Web, Android and iOS). One of its main features is to automatically propagate data changes to clients in real-time or even query the database from the client (well, the client has a cache of the database which will update the remote one). It is an easy framework to build reactive applications

How to deploy on Clever Cloud

As an example of a large Meteor application, I will use TelescopeJS. It's a social news application you can configure the way you want. I'll use the version v0.14.2 since it does not require MongoDB 3.0. This is a version we will provide in a near future.

1- Clone

So first, git clone https://github.com/TelescopeJS/Telescope && cd Telescope && git checkout v0.14.2 and run it: meteor run. Then open http://localhost:3000, you should see the default page with some examples.

2- Install and build

To make it run on Clever Cloud, we have to make our own install script. This script will run when you deploy your application and build what's needed. Create an install.sh file and paste the following lines (or download it from Github):

#! /bin/bash

# Current path
currentPath=$(realpath ./)

# Install Meteorjs
curl https://install.meteor.com/ | sh
export PATH=/home/bas/.meteor:$PATH

# Install demeteorizer
cd ~/ && npm install demeteorizer

# Go back to our project
cd "$currentPath"

# Note: When using "meteorhacks:npm"
# to prevent the error: "unknown package: npm-container"
# (described in https://github.com/meteorhacks/npm/issues/49)
# uncomment the two following lines:
#meteor remove npm-container
#meteor run

# demeteorize the app
~/node_modules/.bin/demeteorizer -a "my_app" -o my_app/

# Go inside our demeteorized app to install modules
cd my_app/

# Install modules
npm install

This script will install Meteorjs on our platform, demeteorize the app (i.e. convert it as a regular node.js application) and install its modules. If you are using the package npm-container from meteorhacks, you HAVE to uncomment lines 20 and 21 (this is not our case here). Do not forget to do a chmod +x install.sh to give it execution rights.

Now we have to create our package.json (which is the file which describe your application for NPM): npm init. Open it and create a scripts section:

"scripts":{
  "install": "./install.sh",
  "start": "node my_app/main.js"
}

(my_app/main.js is a file generated by demeteorizer, do not modify this line)

Then, you can commit your changes: git add package.json install.sh && git commit -m "Clever Cloud setup"

Configuration

Once done, we need to create a Node.js application and a MongoDB addon and configure it. Open the dashboard and:

Addon

• Add an addon
• Select MongoDB, choose your plan and name it.
• Click it in the blue pane on your left, go into its configuration tab and copy somewhere the Connection URI field

Application

• Add an application
• Choose node.js for the language
• Choose the scalers you want (the default configuration is enough most of the time)
• Name it
• We don't need an addon since we've just created it
• Environment variables: You have to create 2 of them: ROOT_URL (url of you application) and MONGO_URL (the Connection URI field you saved)
• Follow instructions to add the remote repository, git push and it will deploy.

Your application should now be deployed, feel free to contact our support if you have any troubles!