Block volumes (RBD) for virtual machines and containers.
Infrastructure
Ceph
We design and run distributed storage with Ceph. The certification is issued by Red Hat, and few companies in Italy hold it.
What Ceph is
A storage system that spreads data across many machines rather than keeping it on one appliance. When a disk or a whole node fails, the data stays available and the system repairs itself. It grows by adding machines, without stopping.
- Type
- Distributed storage, no single point of failure
- Block
- RBD: disks for virtual machines and containers
- File
- CephFS: shared filesystem, mountable from several machines
- Object
- RGW: object storage compatible with S3 and Swift
- Growth
- Nodes added live, without downtime
- Redundancy
- Replication or erasure coding, configurable
- Integration
- Native in Proxmox VE, and on standalone clusters
- Licence
- LGPL: free software
One storage system, many ways to use it
Ceph presents itself to vSphere over iSCSI or NFS: the storage changes, the hypervisor stays.
Amazon S3
Compatible with S3 and Swift: object storage on premise, with the data staying in house.
Kubernetes
Persistent storage over CSI, surviving a pod or a node restarting.
Object Lock
Immutable copies ransomware cannot encrypt, not even with administrator credentials. It works with Veeam.
CephFS
A filesystem that mounts from several machines at once, with no NFS server as a bottleneck.
Snapshots
Point-in-time copies of volumes and filesystems, and clones that start from them.
Remote replication
The data sits in more than one site at once: if one goes, it is already elsewhere.
Archives
From a few TB to tens of petabytes, adding nodes live without stopping anything.
Commodity hardware
Runs on ordinary machines, including servers the manufacturer no longer supports.
How it can be built
Hyper-converged
The same machines do virtualization and storage, inside a single Proxmox VE cluster. Less hardware to buy, to power and to manage.
Dedicated cluster
Ceph on its own machines, separate from compute, bare metal included. Needed when storage has to grow on its own terms, serve several clusters, or take loads that would disturb the virtual machines.
With a SAN interface
When the systems that will use the storage speak iSCSI, SMB or NFS: Windows servers, appliances expecting a conventional SAN: there is PetaSAN, which we distribute in Italy. It is Ceph: same engine, same distributed data, with the protocols those systems already speak on top.
On clusters we did not install, too
It is the common case. You do not have to explain how it was put together or why: that is ours to work out. We are often called in on clusters built by somebody who has since moved on.
What we do
Design
How many nodes, how many disks, what network. Sizing decides whether the cluster holds when a node goes down: it is where mistakes are made, and they surface late.
Deployment
Installation, replication rules, integration with virtualization, and failure testing before anything goes into production.
Maintenance and upgrades
Version upgrades, disk replacement, data rebalancing, expansion with new nodes. A Ceph cluster needs looking after, not installing and forgetting.
Disaster recovery
Degraded clusters, collapsed performance, data that will not come back, an upgrade gone wrong. We work on installations built by others too.
Emergencies
When storage is down and unresponsive, we step in straight away. That is the moment you want someone who has seen it before.
We train the people who will run it
Two levels, from the architecture to diagnosis when something goes wrong. In our classroom, at your site or online, backed by the Red Hat certification.
What we work with
When it makes sense
Ceph is worth it from three nodes up, when data cannot stop and the volume grows over time. Below that there are simpler answers, and we will say so rather than sell you a cluster you do not need.
Questions we get
Do you work on clusters installed by someone else?
Yes, and it is the common case. It does not need to be ours: we look at how it was built, tell you what we see, and you decide from there. We are often called in on clusters put together by someone who has since moved on.
Can you migrate existing storage onto Ceph?
Yes. It is done with the service running, moving the data a piece at a time: virtual machines keep working while the transfer goes on. Any downtime is limited to the final switch-over, and it is agreed with you.
How many nodes do we need to start?
Three is the minimum for the cluster to survive losing a node without stopping. You can start there and add more when you need them, with nothing to rebuild. Below three nodes Ceph makes no sense, and we will say so rather than sell it anyway.
Does it need special hardware?
There is no approved hardware list: Ceph runs on standard equipment, and you are not tied to one supplier. There are characteristics that do matter, though: a dedicated, fast network and disks built to run continuously: because those are what decide whether the cluster performs day to day. It is not about the brand, it is about how it is built: we check that with you before you buy anything.
What happens if a node fails?
Nothing you have to do straight away. The data sits on several nodes, so virtual machines carry on working and the cluster rebuilds the missing copies by itself. The node gets replaced in your own time, not in the middle of the night.
Have storage to design, storage that is misbehaving, or an emergency?
Tell us the situation: how much data, how many nodes, what worries you. We give an opinion before any quote.
If the cluster is down right now, we run a service dedicated to Ceph emergencies, ceph-support.eu: same team, a channel built to answer quickly.