如何安全删除osd

A、普通方式部署的

root@pve-ceph01:~# cat /root/delete_osd.sh

#!/bin/bash
#用法：delete_osd.sh 1 #这个1是 指osd.1

sudo ceph osd out $1
sleep 2
sudo systemctl stop ceph-osd@$1.service
sleep 2
sudo ceph osd crush remove osd.$1
sleep 2
sudo ceph auth del osd.$1
sleep 2
sudo ceph osd rm $1
sleep 2
if [ -d "/var/lib/ceph/osd/ceph-$1" ];then
    sudo umount /var/lib/ceph/osd/ceph-$1
    sleep 2
    sudo rm -rf /var/lib/ceph/osd/ceph-$1
fi

B、pve方式部署的

1、out 对应节点的 osd
    ceph osd out 0    #out id为0的osd

2、stop 对应节点的 osd
    systemctl stop ceph-osd@0        #仅停止osd.0的进程
    #systemctl stop ceph-osd.target  #停止该host上所有的osd进程

3、destroy 对应节点的 osd（不更换的就不用销毁，另开一个终端监控ceph -s）
    pveceph osd destroy 0 -cleanup

附、若是删除后无法新建osd

1、删除osd后，新建osd出现如下报错
root@lahdd004:~#                                                                                 pveceph osd create /dev/sdh -db_dev /dev/nvme1n1 -db_dev_size 30
create OSD on /dev/sdh (bluestore)e /dev/sdh -db_dev /dev/nvme1n1 -db_dev_size 30
creating block.db on '/dev/nvme1n1'
  Logical volume "osd-db-1454dc2e-b0c5-4b40-b11f-d9f3aaeed22c" created.
using 'ceph-ade104a4-3494-4663-b2d7-de6b04569d2b/osd-db-1454dc2e-b0c5-4b40-b11f-d9f3aaeed22c' for block.db
wiping block device /dev/sdh
200+0 records in
200+0 records out
209715200 bytes (210 MB, 200 MiB) copied, 1.38894 s, 151 MB/s
Running command: /bin/ceph-authtool --gen-print-key
Running command: /bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 579b1554-19a5-494e-8ef1-a75270ef5f8f
 stderr: 2023-05-03T19:42:11.706-0700 7fcbdea5c700 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.bootstrap-osd.keyring: (2) No such file or directory
 stderr: 2023-05-03T19:42:11.706-0700 7fcbdea5c700 -1 AuthRegistry(0x7fcbd8059780) no keyring found at /etc/pve/priv/ceph.client.bootstrap-osd.keyring, disabling cephx
 stderr: Error EEXIST: entity osd.130 exists but key does not match
-->  RuntimeError: Unable to create a new OSD id
command 'ceph-volume lvm create --cluster-fsid 3a9af753-3f48-43d8-b0e3-b6a3189f41e7 --block.db ceph-ade104a4-3494-4663-b2d7-de6b04569d2b/osd-db-1454dc2e-b0c5-4b40-b11f-d9f3aaeed22c --data /dev/sdh' failed: exit code 1

2、可以尝试如下解决办法
    ln -s /var/lib/ceph/bootstrap-osd/ceph.keyring /etc/pve/priv/ceph.client.bootstrap-osd.keyring

    或者cp /var/lib/ceph/bootstrap-osd/ceph.keyring /etc/pve/priv/ceph.client.bootstrap-osd.keyring

    这是参考：
        https://forum.proxmox.com/threads/unable-to-create-ceph-osd.56501/
        https://tracker.ceph.com/issues/48392
        https://niziak.spox.org/wiki/vm:proxmox:ceph:performance

3、当然最后我发现，不需要上面的解决方法，而是ceph auth osd.130认证信息没有删除。