Monday 30 November 2015

Unknown filesystem type 'LVM2_member' - Debian

So you are working with LVM volumes specifically with LVM volumes that contain complete filesystems - i.e. virtual hard disks - and you are getting this 'unknown filesystem type 'LVM2_member' when you are trying to mount the volume. This error is encountered because this partition contain multiple LVM parts (i.e. members). The way you would mount these volumes is to extract this partition, mount it as a loop device and then use pvs/lvs and vgchange command to activate the LVM and mount. Consider my LVM volume that exist on a Xen/Ganeti host:
# gnt-instance info migrate.isys

[cut]

  Disk template: plain
  Disks:
    - disk 0: lvm, size 10.0G
      access mode: rw
      logical_id:  volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data
      on primary:  /dev/volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data (254:66)
[cut]
Use kpartx to separate the partitions of the volume:
# kpartx -a -p- /dev/volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data
# ls -la /dev/mapper/volg1-50c29778*
lrwxrwxrwx 1 root root 8 Nov 30 14:46 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data -> ../dm-66
lrwxrwxrwx 1 root root 8 Nov 30 15:30 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-1 -> ../dm-79
lrwxrwxrwx 1 root root 8 Nov 30 15:30 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-2 -> ../dm-80
Let's mount the first partition: /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-1
# mount /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-1 /mnt

# ls -la /mnt
total 34110
dr-xr-xr-x.  5 root root     1024 Nov 23 13:28 .
drwxr-xr-x  23 root root     4096 May 11  2015 ..
-rw-r--r--.  1 root root   107134 Jul 23 17:13 config-2.6.32-573.el6.x86_64
drwxr-xr-x.  3 root root     1024 Nov 23 13:27 efi
drwxr-xr-x.  2 root root     1024 Nov 23 13:29 grub
-rw-------.  1 root root 27635640 Nov 23 13:29 initramfs-2.6.32-573.el6.x86_64.img
drwx------.  2 root root    12288 Nov 23 13:25 lost+found
-rw-r--r--.  1 root root   205998 Jul 23 17:14 symvers-2.6.32-573.el6.x86_64.gz
-rw-r--r--.  1 root root  2585052 Jul 23 17:13 System.map-2.6.32-573.el6.x86_64
-rwxr-xr-x.  1 root root  4220560 Jul 23 17:13 vmlinuz-2.6.32-573.el6.x86_64
-rw-r--r--.  1 root root      166 Jul 23 17:13 .vmlinuz-2.6.32-573.el6.x86_64.hmac
Looks like it's the /boot partition. Fine, the rest of my disk is on the second partition, so let's mount that:
# mount /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-2 /mnt
mount: unknown filesystem type 'LVM2_member'
Uh-oh, can not mount: unknown filesystem type 'LVM2_member'. So the second partition must be made up of more than one LVM members. Lets make an image of this and mount it as a loop back filesystem to see what is inside.
# dd if=/dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-2 of=/var/tmp/partition2.dd bs=1M
10039+0 records in
10039+0 records out
10526654464 bytes (11 GB) copied, 174.754 s, 60.2 MB/s

# file partition2.dd
partition2.dd: LVM2 PV (Linux Logical Volume Manager), UUID: NydiFK-CaAX-PWcz-I0pY-EpAk-4ViC-pIGpf0, size: 10526654464
# losetup /dev/loop0 partition2.dd
# pvs
  PV         VG          Fmt  Attr PSize  PFree
  /dev/loop0 volg0       lvm2 a--   9.78g 992.00m
  /dev/sda5  SYSTEM      lvm2 a--  79.76g  55.55g
  /dev/sdb1  volg1       lvm2 a--   1.97t   1.96t
# lvs
  LV   VG          Attr     LSize  Pool Origin Data%  Move Log Copy%  Convert
  HOME SYSTEM -wi-ao--  3.72g
  ROOT SYSTEM -wi-ao--  7.45g
  SWAP SYSTEM -wi-ao--  3.72g
  TMP  SYSTEM -wi-ao--  1.86g
  VAR  SYSTEM -wi-ao--  7.45g
  root volg0       -wi-----  7.81g
  swap volg0       -wi-----  1.00g
Hey look! It's made up of root and a swap member! Now if you want to mount these, you can now run the vgchange command and activate them:
# vgchange -ay volg0
  2 logical volume(s) in volume group "volg0" now active

# lvdisplay

[snip]

  --- Logical volume ---
  LV Path                /dev/volg0/root
  LV Name                root
  VG Name                volg0
  LV UUID                dAoIl6-WR4G-Z0d1-9Cgg-hevk-oSaf-7TTeGv
  LV Write Access        read/write
  LV Creation host, time migrate.isys.bris.ac.uk, 2015-11-23 13:25:26 +0000
  LV Status              available
  # open                 0
  LV Size                7.81 GiB
  Current LE             250
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:5

  --- Logical volume ---
  LV Path                /dev/volg0/swap
  LV Name                swap
  VG Name                volg0
  LV UUID                L0haeP-jWyr-Gij9-sAhA-8mk6-VGS0-M42Gyl
  LV Write Access        read/write
  LV Creation host, time migrate.isys.bris.ac.uk, 2015-11-23 13:25:30 +0000
  LV Status              available
  # open                 0
  LV Size                1.00 GiB
  Current LE             32
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:6

# mount /dev/volg0/root /mnt
# ls /mnt
bin  boot  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  proc  root  sbin  selinux  srv  sys  tmp  usr  var
And there you are, you have access to your root data! Copy it, move it, do anything you want, then perform the clean up-reverse actions:
# umount /mnt
# vgchange -an volg0
  0 logical volume(s) in volume group "volg0" now active
# losetup -d /dev/loop0
# kpartx -d -p- /dev/volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data
# ls -la /dev/mapper/volg1-50c29778*
lrwxrwxrwx 1 root root 8 Nov 30 14:46 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data -> ../dm-66

Wednesday 25 November 2015

Puppet Error: Error 400 on SERVER: Error from DataBinding 'hiera' while looking up 'puppet_enterprise::certificate_authority_port': (): could not found expected ':'

If you get this Puppet error (when running puppet agent -t on your client): Error 400 on SERVER: Error from DataBinding 'hiera' while looking up 'puppet_enterprise::certificate_authority_port': (): could not found expected ':' It means that there is a syntax error in you hiera yaml file.

Friday 20 November 2015

How to extend root partition in LVM

Follow these instructions! https://rbgeek.wordpress.com/2013/01/27/how-to-extend-the-root-partition-in-lvm/

Wednesday 18 November 2015

Fix grub boot loader - Debian 7

Boot from CDROM, choose Rescue Mode and Advanced options. When prompted, request a shell on the root partition With the grub tools (apt-get install grub if tools are missing):
grub-mkdevicemap
grub-install /dev/sda
update-grub
Reboot

Thursday 12 November 2015

Start service at boot (autostart) - CentOS 6

chkconf is the command, for example auto start Apache at boot:
chkconfig httpd on

Wednesday 11 November 2015

Fixing deleted /var/run link and restoring SELinux permissions - CentOS 7

So some idiot with sudo permissions deleted the symlink /var/run. To restore it, log on to another system and find out the permissions of this link:
# ls -ld /var/run
lrwxrwxrwx. 1 root root 6 Feb 25  2015 /var/run -> ../run
# ls -Z /var/run
lrwxrwxrwx. root root system_u:object_r:var_run_t:s0   /var/run -> ../run
#
Log on to broken system as root and do the following:
# cd /var
# ln -s ../run run
# chcon -h system_u:object_r:var_run_t:s0 run
# ls -Z run
lrwxrwxrwx. root root system_u:object_r:var_run_t:s0   /var/run -> ../run