Storage(SW)
https://www.redhat.com/en/technologies/storage CEPH: Provides a robust, highly scalable block, object, and filesystem storage platform for modern workloads, like cloud infrastructure and data analytics. Consistently ranked as preferred storage option by OpenStack® users. Gluster: Provides a scalable, reliable, and cost-effective data management platform, streamlining file and object access across physical, virtual, and cloud environments.
http://cecs.wright.edu/~pmateti/Courses/7370/Lectures/DistFileSys/distributed-fs.html
| HDFS | iRODS | Ceph | GlusterFS | Lustre |
Arch | Central | Central | Distributed | Decentral | Central |
Naming | Index | Database | CRUSH | EHA | Index |
API | CLI, FUSE | CLI, FUSE | FUSE, mount | FUSE, mount | FUSE |
REST | REST | | REST | | |
Fault-detect | Fully connect. | P2P | Fully connect. | Detected | Manually |
sys-avail | No-failover | No-failover | High | High | Failover |
data-aval | Replication | Replication | Replication | RAID-like | No |
Placement | Auto | Manual | Auto | Manual | No |
Replication | Async. | Sync. | Sync. | Sync. | RAID-like |
Cache-cons | WORM, lease | Lock | Lock | No | Lock |
Load-bal | Auto | Manual | Manual | Manual | No |
http://www.youritgoeslinux.com/impl/storage/glustervsceph
software defined distributed storage that can scale to several petabytes. It provides interfaces for object, block and file storage. http://docs.gluster.org/en/latest/Quick-Start-Guide/Architecture/
http://docs.gluster.org/en/latest/Quick-Start-Guide/Quickstart/
gluster peer probe server2 # from server1gluster peer probe server1 # from server2# Once this pool has been established, it must be probed from the pool.gluster peer statusmkdir -p /data/brick1/gv0 # both server1 and server2gluster volume create gv0 replica 2 server1:/data/brick1/gv0 server2:/data/brick1/gv0 # any servergluster volume start gv0gluster volume info
gluster volume create test-volume \server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4 # Distributedreplica 2 transport tcp server1:/exp1 server2:/exp2 # Replicated, w/ data redundancyreplica 2 transport tcp server1:/exp1 ... server4:/exp4 # R->D
stripe 2 transport tcp server1:/exp1 server2:/exp2 # Stripedstripe 4 transport tcp server1:/exp1 ... server8:/exp8 # S->D
stripe 2 replica 2 transport tcp server1:/exp1 ... server4:/exp4 # R->S （*）stripe 2 replica 2 transport tcp server1:/exp1 ... server8:/exp8 # R->S->D （*）（*） only for Map Reduce workloads
<Dispersed Volume Usable size> = <Brick size> * (#Bricks - Redundancy)bricks#(>=3) > 2 * redundancy[disperse [<count>]] [redundancy <count, default=auto>]disperse 4 server{1..4}:/bricks/test-volume # Disperseddisperse 3 server1:/brick{1..3} # Dispersed (#)disperse 3 server1:/brick{1..6} # Distributed Dispersed (#)(#) on the same server, add `force`
disperse-data 8 redundancy 4 transport tcpserver1:/exp/brick1 server1:/exp/brick2 server1:/exp/brick3 server1:/exp/brick4server2:/exp/brick1 server2:/exp/brick2 server2:/exp/brick3 server2:/exp/brick4server3:/exp/brick1 server3:/exp/brick2 server3:/exp/brick3 server3:/exp/brick4server1:/exp/brick5 server1:/exp/brick6 server1:/exp/brick7 server1:/exp/brick8server2:/exp/brick5 server2:/exp/brick6 server2:/exp/brick7 server2:/exp/brick8server3:/exp/brick5 server3:/exp/brick6 server3:/exp/brick7 server3:/exp/brick8server1:/exp/brick9 server1:/exp/brick10 server1:/exp/brick11 server1:/exp/brick12server2:/exp/brick9 server2:/exp/brick10 server2:/exp/brick11 server2:/exp/brick12server3:/exp/brick9 server3:/exp/brick10 server3:/exp/brick11 server3:/exp/brick12
gluster volume info test-volumegluster volume start test-volume
http://docs.gluster.org/en/latest/Administrator Guide/RDMA Transport/
As of now only FUSE client and gNFS server would support RDMA transport.
http://docs.gluster.org/en/latest/Administrator Guide/Managing Snapshots/ GlusterFS volume snapshot feature is based on thinly provisioned LVM snapshot.
http://docs.gluster.org/en/latest/Administrator Guide/Gluster On ZFS/
Ceph uniquely delivers object, block, and file storage in one unified system. A Ceph Storage Cluster consists of two types of daemons:
Ceph Monitor: maintains a master copy of the cluster map
Ceph OSD Daemon: checks its own state and the state of other OSDs and reports back to monitors.
http://docs.ceph.com/docs/master/cephfs/
STEP 1: METADATA SERVERSTEP 2: MOUNT CEPHFS
http://hub.docker.com/r/ceph/
http://docs.ceph.com/docs/master/rbd/rbd-snapshot/ Ceph supports many higher level interfaces, including QEMU, libvirt, OpenStack and CloudStack. Ceph supports the ability to create many copy-on-write (COW) clones of a block device shapshot. Snapshot layering enables Ceph block device clients to create images very quickly.
https://github.com/inkscope/inkscope (with screenshots)
File, Block, and Object Storage Services for your Cloud-Native Environment
https://rook.github.io/docs/rook/master/kubernetes.html 
https://github.com/ipfs/ipfs
combines Kademlia + BitTorrent + Gitmountable filesystem (via FUSE)http://ipfs.io/<path>
https://github.com/duplicati/duplicati https://www.duplicati.com/screenshots/ Store securely encrypted backups in the cloud! Amazon S3, OneDrive, Google Drive, Rackspace Cloud Files, HubiC, Backblaze (B2), Amazon Cloud Drive (AmzCD), Swift / OpenStack, WebDAV, SSH (SFTP), FTP, and more!
