Wrapping up that Ceph Bluestore conversion...

If you've run Ceph clusters for a long time, you probably either finished or are going through a Filestore to Bluestore conversion. For big clusters with lots of objects, it's a very long process. One of our oldest clusters has been converting for over a year, and it still isn't finished yet. It's a complicated function of object size, number of objects, EC settings, storage speed, etc.

A few places probably built a parallel cluster(s) out and copied the data between them. We've done that in a few places, but there were a few business needs that required conversion in place. For that large cluster, we chose the "whole host replacement" method described here: https://docs.ceph.com/en/latest/rados/operations/bluestore-migration/#convert-existing-osds. This method works, is safe, and it's supported by RedHat if you're under commercial support. There's one problem with that article though. It doesn't tell you how to put your spare host back into the cluster once you're done. It tells you to kick a node out with this command:

One would think it would be as simple as running the opposite, right? Well sorta. The command is indeed "ceph osd crush link", but that command isn't documented well and there are no examples of people using it in search results. So, this blog post is an attempt to seed the search engines.  You would run something like this to get that host back in the default root:

Or put it in a specific rack (like I wanted):