tl;dr: On March 21th 2022 the Cellar C1 cluster will reach its end of life. So we ask you to perform a migration to a newer cluster Cellar, which will also improve performance. The C1 cluster did not accept new add-ons since the 20th January 2019.

In fact, data need to be migrated from this old add-on to a new Cellar add-on. Newly created Cellar C2 add-ons come with various improvements :

• A better S3 protocol support (including the AWS V4 signature)
• Improved upload and download performances (x3 for upload, x2 for download)
• Improved resilience and availability of your data (Data is now replicated across two datacenters more than 20km apart)

How to migrate my data?

We can help you to migrate your data by syncing it directly on our side. To do that, reach the support and include in your message source and destination add-ons, including the buckets to migrate. The support team is here to help.

If you wish to migrate on your side, or at least keep your buckets synchronized, we built a simple tool to help you: Cellar migration tool.
You can run this tool as many times as you need. It will only synchronize objects that are different between the source and the destination buckets.
If you do not feel comfortable using it, you can also use the s3cmd sync command from the s3cmd tool.

If your application handle it, we encourage you to upload new objects as soon as possible to the new bucket. Then when you need to query that object, your application can query both buckets (old and new) to get it. You can also store the complete URL to make this easier. This should help you transition from one bucket to another without downtime.

Please note that we can't make sure that the buckets names you use will be available on the destination cluster. If you have a name conflict, please contact us and we will work something out.

A few days before the deadline, we will have short service brown-outs to remind customers that the service is going to be shut down. Those brown-outs will be announced on our status page a few days before. Make sure to subscribe to Status Updates to receive a notification.

What will happen if I can't do the migration?

If you can't do the migration or have any questions about it, we strongly recommend that you contact our support team via the console.
Before shutting down the cluster, we will backup your data and keep it for 6 months. Our support team will be able to provide you a download link or re-inject the data in a new bucket if you wish to recover it.

Good news everyone, we are really excited to offer the Elastic Stack — 🔥Platinum Version 🔥 — on Clever Cloud. It's the first as a service offer that is officially supported by Elastic from a French company, with datacenters located in France.

If you are looking for an Elastic Stack provider dealing with the American Cloud Act problematics and the GDPR regulations, look no further. :)

We are proud to partner with Elastic and offer you the full Elastic Stack on Clever Cloud.

Am I talking about X-Pack? Yes and no, I am talking about so much more. You will have access to Kibana Canvas & Lens, Kibana Spaces with full security (encryption, RBAC, field and doc-level security), Alerting, Elasticsearch SQL, Machine Learning, Metrics, logs, ... If you want the full details about what is available on Clever Cloud, take a look at the Platinum column on this page.

Our starting price is 17 euros per month. With this you can get the full extent of the Elastic Stack, at a very small scale. On the other hand, you can go all the way up to 64 CPUs and 256Go of RAM per node. Our support team will manage the first levels of support while being able to escalate to Elastic's team when needed. You are in good hands!

And of course we worked on an integration with the rest of Clever Cloud. Please have a look at our documentation for the details. Here's a quick glance at what we did.

Specific Clever Cloud Integrations

We have worked on our Elastic Stack integration on several fronts. When you provision the Elastic add-on, we allow you to provision Kibana and an APM server as traditional Clever Cloud applications. It means that they benefit from all the goodness that we bring to applications. You can turn them off if you want to, you can enable auto scalability, you can link them to other applications, really anything you would do with traditional applications. Let's see each integration a bit more in details.

Authentication

Authenticating to Kibana is available through an automatically configured SSO. Every member of the organisation the addon has been deployed to can use their Clever Cloud account to authenticate. No configuration is required on your part.

Elastic APM

If you link the APM server application to any of your application, the right environment variables will be injected and automatically picked by the APM agent in your dependencies. Then simply authenticate to Kibana and start setting up your APM! This feature will be showcased in a dedicated blog post in the coming days.

Backups

We are introducing a new way to manage your add-on backups. When we create your new add-on, we also create a Cellar add-on (our S3-compatible object storage solution) named Backups. All your backups will be stored there. We are starting with Elasticsearch but other databases will soon follow.

What's next?

We plan to provide an even better integration with the Elastic Stack. We are currently thinking about the best way to integrate Beats to our applications. You can currently do it manually but wouldn't it be nice if this was automated? This should really ease the usage of Elastic SIEM for instance.

And of course we are working on automatic cluster provisioning, not just for Elastic. In the meantime you can provision as many nodes as you need and contact our support team to put them in the same cluster.

We are supper excited about working with Elastic and hope you will be as excited to try it on Clever Cloud! We will publish more blog posts in the coming days, highlighting some of the awesome capabilities of our Elastic Stack integration.

➡️ https://console.clever-cloud.com

If you’re used to code micro-services with Node.js, you may be familiar with the frustration of performance issues, finding the root cause of an error and hunting down memory leaks.

That’s why services like Trace are here. Trace is a Node.js performance monitoring tool which helps you to understand how your application behaves and lets you find performance bottlenecks with ease.

Today we are launching Trace as an add-on for Clever Cloud, which can be tried out for free until July 1st without any restrictions.

Distributed tracing

Trace comes with a unique feature called distributed tracing: It visualizes whole transactions (request-chains) in your microservices architecture and looks for errors.

This means that you can see on an interactive timeline:

• which services were taking part in a transaction,
• how big were the response network delays,
• how long did a service handled a request,
• where and when the errors happened in that transaction.

Trace connects services taking part in a request by attaching correlation IDs to them. This way you can visualize the exact data-flow of faulty transactions, see the dependencies between your microservices, look for bad status codes and localize ongoing issues.

Service mapping

Trace provides you with a dynamic service map, which is automatically generated based on how the services in your system communicate with each other, or with your databases and external APIs.

Thanks to this, you can see how your application really looks like, and understand what makes it to slow down. The service map also allows you to find out how many requests your services handle and how big are their response times.

Getting started

Adding Trace to your services is possible with just a couple lines of code, and it can be installed and used in under two minutes. There is a tutorial for the installation on Trace’s documentation: http://trace-docs.risingstack.com/docs/fix-crashing-nodejs-applications-with-trace

Some Use cases

Using Trace to fix crashing Node.js apps: http://trace-docs.risingstack.com/docs/fix-crashing-nodejs-applications-with-trace

Using Trace to investigate slow Node.js apps: http://trace-docs.risingstack.com/docs/fix-crashing-nodejs-applications-with-trace

Storing files and assets on Clever Cloud just got easier. After a year of beta testing, we’re proud to release Cellar: a S3-compatible file storage API.

Cellar is faster, cheaper and more reliable thanks to its distributed architecture.

If you’re using FS buckets or storing files in your database, switching to Cellar will make your life easier: files can be directly uploaded to and downloaded from Cellar, without ever going through your application. This way you can handle large files without a hassle.

Cellar is perfect for:

• user avatars
• large files (images, videos)

Pricing

Storage

• First 100MB Free
• First 1TB € 20.48 / TB / mo
• First 25TB € 15.36 / TB / mo
• First 50TB € 10.24 / TB / mo

Traffic (outbound)

• First 100MB Free
• First 10TB € 0.09 / GB / mo
• First 40TB € 0.07 / GB / mo

Getting started with Cellar and s3cmd

Create a new Cellar addon in the Clever Cloud console. From the addon dashboard, you will be able to get your access credentials, as well as to download a s3cmd configuration file to manage your files on Cellar.

Create a bucket

In cellar, files are stored in buckets, accessible under bucket-name.cellar.services.clever-cloud.com. To start using cellar, you first have to create a bucket for your files.

You can create a bucket with s3cmd (make sure you have saved the s3cmd configuration file under ~/.s3cfg).

s3cmd mb s3://my-bucket

Upload a file

# private file
s3cmd put my-private-file.jpg s3://my-bucket/my-private-file.jpg

# publicly accessible file
s3cmd put -P my-public-file.jpg s3://my-bucket/my-public-file.jpg

Using Cellar from Node JS

You can use Cellar from any S3-compatible SDK (including the official AWS SDKs) or directly through its REST API.

require fs from 'fs';
require AWS from 'aws-sdk';

// Initialize Cellar Credentials
AWS.config.update({accessKeyId: '<access key>', secretAccessKey: '<access secret>'});
const ep = new AWS.Endpoint('cellar.services.clever-cloud.com');
const s3 = new AWS.S3({ endpoint: ep });


// Upload a file to Cellar
const body = fs.createReadStream('./my-file.jpg');
s3.putObject({Bucket: 'my-bucket', Key: 'my-file.jpg', Body: body}, (err, data) => {
  if (err) console.error(err);
  else console.log("Successfully uploaded data to my-bucket/my-file.jpg");
});

// Get an access URL for a private file
s3.getSignedUrl('getObject', {Bucket: 'my-bucket', Key: 'my-private-file'}, (err, url) => {
  if(err) console.log(err);
  else console.log("The URL is", url);
});

More information

Clever Cloud Cellar is backed by the rock solid Riak Simple Storage (aka Riak S2) from Basho.

Talk about cloud file storage: slides from our CTO Documentation: www.clever.cloud/developers/addons/cellar/

Using Cellar from Node.js with AWS SDK: SDK documentation

Cellar's S3 compatible API: Riak S2 API documentation