【记录】LVM灾难恢复之（一）lvm底层pv成员缺失，如何尽可能的恢复数据

背景介绍（搬迁）

背景介绍（下图是迁移示意图--这里用实验环境进行模拟，每个盘8G）
  1、旧服务器上有 由两块1T的 物理磁盘 组成的 一个lv卷，存放着 图片数据
  2、计划将旧服务器上组成lv卷的那两块磁盘，打标后（sdb,sdc） 整体搬迁 至 新服务器上
  3、由于 搬运者不太靠谱，把 sdc盘丢黄河里了，无奈数据磁盘仅存 sdb插入 新服务器的 2号槽位
  4、老的环境是 ，两个1T 磁盘组成的vg，在上面划的 lv 是1.2T ，据客户反馈，他实际用了差不多 800G空间

需求分析介绍：（数据还是比较重要的，希望尽力找回数据）

1、难点是丢了一个盘，而且是lvm分区形式，会导致无法激活卷组，就无法挂载
2、隐藏特性（恢复思路）：
    A、lvm 默认写入方式是 条带化 形式，（即 数据是挨着挨着写的，正常情况下 800G 其实 只会实际在 老硬盘 sdb上存着），也就是说你要丢了sdb 那就凉凉了
    B、多块磁盘 组成lvm，其中每块磁盘都会存放 一份 lvm配置信息。（也就是说丢失了一块盘，单可以用另一块盘来查看 之前的 lvm配置信息）
    C、 当 一块 带有 lvm 信息的磁盘 插在一台 安装有 lvm软件包的 linux 系统上 ，并经过了一次重启后，
          这台linux 系统自动会把 这块磁盘所携带的 lvm 信息 进行备份，备份在/etc/lvm的相关目录里面（当然你不重启，直接导出lvm配置到指定位置也是可以的）
          /etc/lvm/archive/             下存放着VG及LV调整前的归档信息，即VG变更或LV变更前均会备份当前信息。
    D、注意：使用vgcreate、vgreduce、vgremove、lvcreate、lvreduce、lvremove    等命令时均会生成新的归档信息哦

恢复步骤一：将仅存的一块插入新服务器，可以看到vg组成员有缺失，缺失的成员大小是 8G，并且还有 4G没有划分出去

由于新服务器之前安装系统是采用的标准分区，最小化安装系统所以不会有 lvm软件包
#A、在新服务器上安装lvm2
    yum install lvm2 -y

lvm2 安装完成后，虽然您能通过命令查看到 插入的sdb 所携带的 lvm 相关信息，但是 你却不知道，sdb上的 lvm的配置文件
#B、查看相关的lvm配置
[root@localhost ~]# lvscan 
  WARNING: Device for PV iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA not found or rejected by a filter.
  Couldn't find device with uuid iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA.
  inactive          '/dev/FNK/LT' [12.00 GiB] inherit

[root@localhost ~]# pvscan 
  WARNING: Device for PV iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA not found or rejected by a filter.
  Couldn't find device with uuid iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA.
  PV /dev/sdb   VG FNK             lvm2 [<8.00 GiB / 0    free]
  PV [unknown]  VG FNK             lvm2 [<8.00 GiB / 3.99 GiB free]
  Total: 2 [15.99 GiB] / in use: 2 [15.99 GiB] / in no VG: 0 [0   ]

[root@localhost ~]# vgscan 
  Reading volume groups from cache.
  WARNING: Device for PV iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA not found or rejected by a filter.
  Couldn't find device with uuid iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA.
  Found volume group "FNK" using metadata type lvm2

[root@localhost backup]# pvdisplay 
  WARNING: Device for PV iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA not found or rejected by a filter.
  Couldn't find device with uuid iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA.
  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               FNK
  PV Size               8.00 GiB / not usable 4.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              2047
  Free PE               0
  Allocated PE          2047
  PV UUID               NACDEl-ZZjS-IRaO-cOfC-SeSa-tjrw-oyWCJm

  --- Physical volume ---
  PV Name               [unknown]
  VG Name               FNK
  PV Size               8.00 GiB / not usable 4.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              2047
  Free PE               1022
  Allocated PE          1025
  PV UUID               iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA

[root@localhost backup]# vgdisplay 
  WARNING: Device for PV iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA not found or rejected by a filter.
  Couldn't find device with uuid iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA.
  --- Volume group ---
  VG Name               FNK
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                1
  VG Size               15.99 GiB
  PE Size               4.00 MiB
  Total PE              4094
  Alloc PE / Size       3072 / 12.00 GiB
  Free  PE / Size       1022 / 3.99 GiB
  VG UUID               sb9f6E-tlhF-PqmE-F3Te-IqOa-RFH4-NcD3jQ

恢复步骤二：找一块同型号同品牌的硬盘 插入新服务器，并且重启系统

重启后你可以在/etc/lvm/backup 目录里面看到 之前由 老硬盘 sdb 的生成的 lvm 备份配置文件

[root@localhost backup]# cat /etc/lvm/backup/FNK
# Generated by LVM2 version 2.02.186(2)-RHEL7 (2019-08-27): Wed Jul  1 13:46:42 2020

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'vgscan'"

creation_host = "localhost.localdomain" # Linux localhost.localdomain 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64
creation_time = 1593625602  # Wed Jul  1 13:46:42 2020

FNK {
    id = "sb9f6E-tlhF-PqmE-F3Te-IqOa-RFH4-NcD3jQ"
    seqno = 4
    format = "lvm2"         # informational
    status = ["RESIZEABLE", "READ", "WRITE"]
    flags = []
    extent_size = 8192      # 4 Megabytes
    max_lv = 0
    max_pv = 0
    metadata_copies = 0

    physical_volumes {

        pv0 {
            id = "NACDEl-ZZjS-IRaO-cOfC-SeSa-tjrw-oyWCJm"
            device = "/dev/sdb" # Hint only

            status = ["ALLOCATABLE"]
            flags = []
            dev_size = 16777216 # 8 Gigabytes
            pe_start = 2048
            pe_count = 2047 # 7.99609 Gigabytes
        }

        pv1 {
            id = "iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA"
            device = "[unknown]"    # Hint only

            status = ["ALLOCATABLE"]
            flags = ["MISSING"]
            dev_size = 16777216 # 8 Gigabytes
            pe_start = 2048
            pe_count = 2047 # 7.99609 Gigabytes
        }
    }

    logical_volumes {

        LT {
            id = "uQd7Z0-2xvU-RD9s-hOem-fPdE-YA1f-Gcd4xw"
            status = ["READ", "WRITE", "VISIBLE"]
            flags = []
            creation_time = 1593621196  # 2020-07-01 12:33:16 -0400
            creation_host = "localhost.localdomain"
            segment_count = 2

            segment1 {
                start_extent = 0
                extent_count = 2047 # 7.99609 Gigabytes

                type = "striped"
                stripe_count = 1    # linear

                stripes = [
                    "pv0", 0
                ]
            }
            segment2 {
                start_extent = 2047
                extent_count = 1025 # 4.00391 Gigabytes

                type = "striped"
                stripe_count = 1    # linear

                stripes = [
                    "pv1", 0
                ]
            }
        }
    }

}

恢复步骤三：

将 新硬盘 sdc 预置为 pv，并修改 原本的 lvm配置文件/etc/lvm/backup/FNK，把 unknown的那个pv成员 信息用新 硬盘的信息 进行 填充（新的pv uuid 和状态flags）

A、先备份 老的 lvm配置文件
    cp /etc/lvm/backup/FNK /etc/lvm/backup/FNK.bak

B、将 新的sdc硬盘 预置为 pv，并记录其 pv uuid
    [root@localhost home]# pvcreate /dev/sdc
  Physical volume "/dev/sdc" successfully created.

[root@localhost home]# pvdisplay 
  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               FNK
  PV Size               8.00 GiB / not usable 4.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              2047
  Free PE               0
  Allocated PE          2047
  PV UUID               NACDEl-ZZjS-IRaO-cOfC-SeSa-tjrw-oyWCJm

  "/dev/sdc" is a new physical volume of "8.00 GiB"
  --- NEW Physical volume ---
  PV Name               /dev/sdc
  VG Name               
  PV Size               8.00 GiB
  Allocatable           NO
  PE Size               0   
  Total PE              0
  Free PE               0
  Allocated PE          0
  PV UUID               4YCs0F-2282-Cxei-9WYR-esCX-HeS7-K9NFor

C、把 unknown的那个pv成员 信息用新 硬盘的信息 进行 填充 (pv uuid 以及 flags 状态)
[root@localhost home]# cat /etc/lvm/backup/FNK
# Generated by LVM2 version 2.02.186(2)-RHEL7 (2019-08-27): Wed Jul  1 13:46:42 2020

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'vgscan'"

creation_host = "localhost.localdomain" # Linux localhost.localdomain 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64
creation_time = 1593625602  # Wed Jul  1 13:46:42 2020

FNK {
    id = "sb9f6E-tlhF-PqmE-F3Te-IqOa-RFH4-NcD3jQ"
    seqno = 4
    format = "lvm2"         # informational
    status = ["RESIZEABLE", "READ", "WRITE"]
    flags = []
    extent_size = 8192      # 4 Megabytes
    max_lv = 0
    max_pv = 0
    metadata_copies = 0

    physical_volumes {

        pv0 {
            id = "NACDEl-ZZjS-IRaO-cOfC-SeSa-tjrw-oyWCJm"
            device = "/dev/sdb" # Hint only

            status = ["ALLOCATABLE"]
            flags = []
            dev_size = 16777216 # 8 Gigabytes
            pe_start = 2048
            pe_count = 2047 # 7.99609 Gigabytes
        }

#      pv1 {
#          id = "iKQcoX-1sJA-xbQw-h0iw-3Ajm-zZI6-vcEIHA"
#          device = "[unknown]"    # Hint only
#          status = ["ALLOCATABLE"]
#          flags = ["MISSING"]
#          dev_size = 16777216 # 8 Gigabytes
#          pe_start = 2048
#          pe_count = 2047 # 7.99609 Gigabytes
#      }
#      下面是替换的信息
        pv1 {
            id = "4YCs0F-2282-Cxei-9WYR-esCX-HeS7-K9NFor"
            device = "[/dev/sdc]"   # Hint only
            status = ["ALLOCATABLE"]
            flags = []
            dev_size = 16777216 # 8 Gigabytes
            pe_start = 2048
            pe_count = 2047 # 7.99609 Gigabytes
        }
    }

    logical_volumes {

        LT {
            id = "uQd7Z0-2xvU-RD9s-hOem-fPdE-YA1f-Gcd4xw"
            status = ["READ", "WRITE", "VISIBLE"]
            flags = []
            creation_time = 1593621196  # 2020-07-01 12:33:16 -0400
            creation_host = "localhost.localdomain"
            segment_count = 2

            segment1 {
                start_extent = 0
                extent_count = 2047 # 7.99609 Gigabytes

                type = "striped"
                stripe_count = 1    # linear

                stripes = [
                    "pv0", 0
                ]
            }
            segment2 {
                start_extent = 2047
                extent_count = 1025 # 4.00391 Gigabytes

                type = "striped"
                stripe_count = 1    # linear

                stripes = [
                    "pv1", 0
                ]
            }
        }
    }

}

恢复步骤四：使用新文件进行恢复 ，并验证数据是否可用

拓展点：跟本次实验无关（执行恢复前lvm 状态一定要是非激活状态）
#A、执行恢复动作，并激活vg组，进行挂载验证
[root@localhost home]# vgcfgrestore -f /etc/lvm/backup/FNK FNK
  Restored volume group FNK
  Scan of VG FNK from /dev/sdb found mda_checksum 4f481a86 mda_size 1320 vs previous e73d2084 1044
  Scan of VG FNK from /dev/sdc found mda_checksum 4f481a86 mda_size 1320 vs previous e73d2084 1044
[root@localhost home]# echo $?      #确认执行成功
0

[root@localhost home]# lvscan 
  inactive          '/dev/FNK/LT' [12.00 GiB] inherit
[root@localhost home]# vgchange -ay
  1 logical volume(s) in volume group "FNK" now active

[root@localhost home]# mount /dev/FNK/LT /tmp/
[root@localhost home]# cd /tmp/
[root@localhost tmp]# ls 
ifcfg-enp0s3  ifdown-eth   ifdown-post    ifdown-Team      ifup-aliases  ifup-ipv6   ifup-post    ifup-Team      init.ipv6-global
ifcfg-lo      ifdown-ippp  ifdown-ppp     ifdown-TeamPort  ifup-bnep     ifup-isdn   ifup-ppp     ifup-TeamPort  network-functions
ifdown        ifdown-ipv6  ifdown-routes  ifdown-tunnel    ifup-eth      ifup-plip   ifup-routes  ifup-tunnel    network-functions-ipv6
ifdown-bnep   ifdown-isdn  ifdown-sit     ifup             ifup-ippp     ifup-plusb  ifup-sit     ifup-wireless
[root@localhost tmp]# cat ifcfg-lo 
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
# If you're having problems with gated making 127.0.0.0/8 a martian,
# you can change this to something else (255.255.255.255, for example)
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback

#B、lvm 恢复成功，数据可以读取。恢复成功