GlusterFS : Distributed + Replication

Configure Storage Clustering with GlusterFS. For example, Create a Distributed + Replication volume with 6 Nodes. Also Configure Arbiter Volume in order to avoid Split Brain. | +----------------------+ | +----------------------+ | [GlusterFS Server#1] |10.0.0.51 | 10.0.0.54| [GlusterFS Server#4] | | node01.srv.world +----------+----------+ node04.srv.world | | | | | | +----------------------+ | +----------------------+ | +----------------------+ | +----------------------+ | [GlusterFS Server#2] |10.0.0.52 | 10.0.0.55| [GlusterFS Server#5] | | node02.srv.world +----------+----------+ node05.srv.world | | | | | | +----------------------+ | +----------------------+ | +----------------------+ | +----------------------+ | [GlusterFS Server#3] |10.0.0.53 | 10.0.0.56| [GlusterFS Server#6] | | node03.srv.world +----------+----------+ node06.srv.world | | | | | +----------------------+ +----------------------+ ⇑ ⇑ file1, file3 ... file2, file4 ...

	It is strongly recommended to use partitions for GlusterFS volumes that are different from the / partition. On this example, it shows settings on the environment that all nodes has [sdb1] and mount it to [/glusterfs].
[1]	Install GlusterFS Server on All Nodes, refer to here.
[2]	Create a Directory for GlusterFS Volume on all Nodes.

root@node01:~#  mkdir -p /glusterfs/dist-replica

[3]	Configure Clustering like follows on a node. (it's OK on any node)

# probe nodes root@node01:~#  gluster peer probe node02 peer probe: success. root@node01:~#  gluster peer probe node03 peer probe: success. root@node01:~#  gluster peer probe node04 peer probe: success. root@node01:~#  gluster peer probe node05 peer probe: success. root@node01:~#  gluster peer probe node06 peer probe: success. # confirm status root@node01:~#  gluster peer status Number of Peers: 5 Hostname: node02 Uuid: f11a43b8-6d3d-4ae7-92ed-1f633699515c State: Peer in Cluster (Connected) Hostname: node03 Uuid: db6c7053-1ac4-4788-b91c-92a73cbb05e3 State: Peer in Cluster (Connected) Hostname: node04 Uuid: d6501012-8698-47c9-9fcf-098c1796c842 State: Peer in Cluster (Connected) Hostname: node05 Uuid: a48a638a-3772-450e-901f-bc7d303a1eee State: Peer in Cluster (Connected) Hostname: node06 Uuid: 1a2e13ae-d98a-4bdc-8b78-4006d6b0d0d8 State: Peer in Cluster (Connected) # create volume root@node01:~#  gluster volume create vol_dist-replica replica 3 arbiter 1 transport tcp \ node01:/glusterfs/dist-replica \ node02:/glusterfs/dist-replica \ node03:/glusterfs/dist-replica \ node04:/glusterfs/dist-replica \ node05:/glusterfs/dist-replica \ node06:/glusterfs/dist-replica volume create: vol_dist-replica: success: please start the volume to access data # start volume root@node01:~#  gluster volume start vol_dist-replica volume start: vol_dist-replica: success # confirm volume info root@node01:~#  gluster volume info Volume Name: vol_dist-replica Type: Distributed-Replicate Volume ID: 06b86241-7040-4f2b-a7fe-c09d6fe9ce5b Status: Started Snapshot Count: 0 Number of Bricks: 2 x (2 + 1) = 6 Transport-type: tcp Bricks: Brick1: node01:/glusterfs/dist-replica Brick2: node02:/glusterfs/dist-replica Brick3: node03:/glusterfs/dist-replica (arbiter) Brick4: node04:/glusterfs/dist-replica Brick5: node05:/glusterfs/dist-replica Brick6: node06:/glusterfs/dist-replica (arbiter) Options Reconfigured: cluster.granular-entry-heal: on storage.fips-mode-rchecksum: on transport.address-family: inet nfs.disable: on performance.client-io-threads: off

[4]	To mount GlusterFS volume on Client Hosts, refer to here.