Create a Ceph Cluster with DeepSea

How to setup a ceph cluster with DeepSea on top of SUSE Leap VMs.

1. Install three VMs and configure "/etc/hosts" -> easiest way to do is to add all nodes in "/etc/hosts" on all VMs. Make sure FQDN are set, too

2. Install "salt-master" on one VM "zypper in salt-master". Enable the service: "systemctl enable salt-master"

3. Install "salt-minion" on ALL VMs "zypper in salt-minion". Enable the service: "systemctl enable salt-minion"

4. Configure "/etc/salt/minion" on all systems to point to the right master and restart minions "systemctl restart salt-minion"

5. Accept keys on the salt-master "salt-key -A -y"

6. Install the build tools on the salt-master VM "zypper in git make"

7. Clone the DeepSea repo "git clone https://github.com/SUSE/DeepSea.git" on the salt-master VM

8. Inside the DeepSea folder run "make install"

9. On the salt master run "systemctl restart salt-master"

10. Run "chown -R salt /srv/pillar/*"

11. Run "salt-run state.orch ceph.stage.0" or "salt-run state.orch ceph.stage.prep"

12. Run "salt-run state.orch ceph.stage.1" or "salt-run state.orch ceph.stage.discovery"

13. Create "/srv/pillar/ceph/proposals/policy.cfg" and run "chown salt:salt /srv/pillar/ceph/proposals/policy.cfg". Example:

    # Cluster assignment
    cluster-ceph/cluster/*.sls
    # Hardware Profile
    profile-*-1/cluster/*.sls
    profile-*-1/stack/default/ceph/minions/*yml
    # Common configuration
    config/stack/default/global.yml
    config/stack/default/ceph/cluster.yml
    # Role assignment
    role-master/cluster/*.sls
    role-mon/cluster/*.sls
    role-mon/stack/default/ceph/minions/*.yml
    

14. Run "salt-run state.orch ceph.stage.2" or "salt-run state.orch ceph.stage.configure"

15. Run "DEV_ENV=true salt-run state.orch ceph.stage.3" or "DEV_ENV=true salt-run state.orch ceph.stage.deploy" -> "DEV_ENV is only needed if you have <=3 VMs"

16. You should change the pg(p)_num to get rid of the "to few pgs per osd" error "ceph osd pool set rbd pg_num 120" and "ceph osd pool set rbd pgp_num 120"

You now should have an up and running cluster.

Now you could add the openATTIC OBS (openSUSE Build Service) repo and install openATTIC and the openattic-deployment package

17. Add the needed Repo from OBS "zypper ar http://download.opensuse.org/repositories/filesystems:/openATTIC/openSUSE_Leap_42.2/filesystems:openATTIC.repo"
18. Install openATTIC "zypper in openattic openattic-module-ceph-deployment" -> you have to break dependencies because deepsea isn't shipped with leap by default
19. Run "chmod 655 /etc/ceph/ceph.*"
20. Run "openattic install"

You're done :)

Update 2017-04-07

• Fixed some typos and added some more steps to do.