Ceph 存储一个重要的使用场景就是其块设备，本文将会介绍一下如何挂载 Ceph RBD 到远程服务器上。从安全角度考虑，一般不建议这样做。

Table of Contents

  1. 准备工作
  2. 远程服务器挂载
    1. 创建 RBD
    2. Ceph 配置
    3. Ceph RBD Map
    4. 格式化与挂载
    5. 自动挂载

准备工作

Ceph 安装 repo:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[ceph]
name=Ceph packages for $basearch
baseurl=https://download.ceph.com/rpm-mimic/el7/$basearch
enabled=1
priority=2
gpgcheck=1
gpgkey=https://download.ceph.com/keys/release.asc

[ceph-noarch]
name=Ceph noarch packages
baseurl=https://download.ceph.com/rpm-mimic/el7/noarch
enabled=1
priority=2
gpgcheck=1
gpgkey=https://download.ceph.com/keys/release.asc

[ceph-source]
name=Ceph source packages
baseurl=https://download.ceph.com/rpm-mimic/el7/SRPMS
enabled=0
priority=2
gpgcheck=1
gpgkey=https://download.ceph.com/keys/release.asc

安装 Ceph 和 RBD 客户端:

1
$ sudo yum install -y ceph-common python-rbd

远程服务器挂载

创建 RBD

1
$ rbd create --pool volulmes --size 50G lintao_test

Ceph 配置

/etc/ceph/ceph.conf:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[global]
fsid = 1da65d0e-4284-41e0-868e-8c25b89648fc
mon_initial_members = ceph01, ceph02, ceph03
mon_host = 192.168.15.38,192.168.15.39,192.168.15.40
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx


[mon]
mgr initial modules = dashboard

[client.cinder]
keyring = /etc/ceph/ceph.client.cinder.keyring

远程客户端使用 key: /etc/ceph/ceph.client.rbd.keyring:

1
2
[client.rbd]
	key = AQBooSldwwbkAhAAtxRH0lHGfnUautW2rF6pkQ==

Ceph RBD Map

  1. map rbd
1
2
$ sudo rbd map volumes/lintao_test --id cinder
/dev/rbd0
  1. if unmap rbd
1
$ sudo rbd unmap volumes/lintao_test --id cinder

格式化与挂载

1
2
3
4
5
6
$ sudo fdisk /dev/rbd/volumes/lintao_test
...
$ sudo mkfs.ext4 /dev/rbd/volumes/lintao_test-part1
...
$ sudo mkdir /data 
$ sudo mount /dev/rbd/volumes/lintao_test-part1 /data

自动挂载

  1. /etc/ceph/rbdmap
1
volumes/lintao_test id=cinder,keyring=/etc/ceph/ceph.client.keyring
  1. enable rbdmap service
1
$ sudo systemctl enable --now rbdmap
  1. /etc/fstab
1
2
3
$ sudo blkid /dev/rbd/volumes/lintao_test-part1
### /etc/fstab
UUID=0a9d74d8-8f37-4a80-ac70-28c292e79784    /data    ext4 defaults 0 0