Resizing VM system disk - LVM Partition 1 does not start on physical sector boundary - CentOS 7

So you have a virtual machine and you've just expanded your virtual disk, log on to your CentOS Linux system and performed a fdisk -l to find this error:

# fdisk -l

Disk /dev/sda: 236.2 GB, 236223201280 bytes, 461373440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1   209715199   104857599+  ee  GPT
Partition 1 does not start on physical sector boundary.

Disk /dev/mapper/centos-root: 34.4 GB, 34359738368 bytes, 67108864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-swap: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-tmp: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-var: 34.4 GB, 34359738368 bytes, 67108864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-opt: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-home: 7507 MB, 7507804160 bytes, 14663680 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Knowing that you have an /dev/sda2 and /dev/sda3 and want to create an /dev/sda4 but how? Install parted, and use this to fix the disk you've just expanded:

# parted
GNU Parted 3.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Error: The backup GPT table is not at the end of the disk, as it should be.  This might mean that another operating system believes the disk is smaller.  Fix, by
moving the backup to the end (and removing the old backup)?
Fix/Ignore/Cancel? Fix
Warning: Not all of the space available to /dev/sda appears to be used, you can fix the GPT to use all of the space (an extra 251658240 blocks) or continue with the
current setting?
Fix/Ignore? Fix
Model: Msft Virtual Disk (scsi)
Disk /dev/sda: 236GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size   File system  Name                  Flags
 1      1049kB  538MB   537MB  ext4
 2      538MB   1075MB  537MB  fat16        EFI System Partition  boot
 3      1075MB  107GB   106GB                                     lvm

(parted) q

Now you can see all three partitions and so will be able to create the fourth:

# fdisk /dev/sda
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p

Disk /dev/sda: 236.2 GB, 236223201280 bytes, 461373440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name
 1         2048      1050623    512M  Microsoft basic
 2      1050624      2099199    512M  EFI System      EFI System Partition
 3      2099200    209713151     99G  Linux LVM

Command (m for help): n
Partition number (4-128, default 4):
First sector (34-461373406, default 209713152):
Last sector, +sectors or +size{K,M,G,T,P} (209713152-461373406, default 461373406):
Created partition 4


Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

Now you have created the forth partition, you can see it with the fdisk -l command:

[root@troi st14988]# fdisk -l
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sda: 236.2 GB, 236223201280 bytes, 461373440 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name
 1         2048      1050623    512M  Microsoft basic
 2      1050624      2099199    512M  EFI System      EFI System Partition
 3      2099200    209713151     99G  Linux LVM
 4    209713152    461373406    120G  Linux filesyste

Disk /dev/mapper/centos-root: 34.4 GB, 34359738368 bytes, 67108864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-swap: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-tmp: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-var: 34.4 GB, 34359738368 bytes, 67108864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-opt: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/centos-home: 7507 MB, 7507804160 bytes, 14663680 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Once you have rebooted for the new partition table to take effect, you can add the new partition to the LVM volume group to extend the logical volumes you need to grow/resize:

# vgs
  VG     #PV #LV #SN Attr   VSize  VFree
  centos   1   6   0 wz--n- 99.00g 4.00m

# vgextend centos /dev/sda4
  Physical volume "/dev/sda4" successfully created
  Volume group "centos" successfully extended

# vgs
  VG     #PV #LV #SN Attr   VSize   VFree
  centos   2   6   0 wz--n- 218.99g 120.00g

Now that you have more space, we can extend any of our logical volumes we want:

# lvs
  LV   VG     Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home centos -wi-ao----  6.99g
  opt  centos -wi-ao---- 20.00g
  root centos -wi-ao---- 32.00g
  swap centos -wi-ao----  4.00g
  tmp  centos -wi-ao----  4.00g
  var  centos -wi-ao---- 32.00g

# lvextend -L +100G centos/opt
  Size of logical volume centos/opt changed from 20.00 GiB (5120 extents) to 120.00 GiB (30720 extents).
  Logical volume opt successfully resized.

# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home centos -wi-ao----   6.99g
  opt  centos -wi-ao---- 120.00g
  root centos -wi-ao----  32.00g
  swap centos -wi-ao----   4.00g
  tmp  centos -wi-ao----   4.00g
  var  centos -wi-ao----  32.00g

# lvextend centos/home -l+100%FREE
  Size of logical volume centos/home changed from 6.99 GiB (1790 extents) to 26.99 GiB (6910 extents).
  Logical volume home successfully resized.

# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  home centos -wi-ao----  26.99g
  opt  centos -wi-ao---- 120.00g
  root centos -wi-ao----  32.00g
  swap centos -wi-ao----   4.00g
  tmp  centos -wi-ao----   4.00g
  var  centos -wi-ao----  32.00g

MongoDB won't start - ERROR: dbpath (/data/db) does not exist.

For some reason - due to a bug probably - mongodb (or mongod) database won't start on default install because it is referencing the running direction /data/db to create a lock file. This error occurs even if you set the dbpath in /etc/mongod.conf to somewhere else (default installation says: /var/lib/mongodb). To fix this, simply create a simlink or the directory /data/db itself!

# service mongod start
Redirecting to /bin/systemctl start  mongod.service
Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.
# systemctl status mongod.service
● mongod.service - High-performance, schema-free document-oriented database
   Loaded: loaded (/usr/lib/systemd/system/mongod.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2016-03-03 09:50:01 GMT; 6s ago
  Process: 21502 ExecStart=/usr/bin/mongod $OPTIONS run (code=exited, status=45)

Mar 03 09:50:01 p0 systemd[1]: Starting High-performance, schema-free document-oriented database...
Mar 03 09:50:01 p0 mongod[21502]: about to fork child process, waiting until server is ready for connections.
Mar 03 09:50:01 p0 mongod[21502]: forked process: 21504
Mar 03 09:50:01 p0 mongod[21502]: ERROR: child process failed, exited with error number 45
Mar 03 09:50:01 p0 systemd[1]: mongod.service: control process exited, code=exited status=45
Mar 03 09:50:01 p0 systemd[1]: Failed to start High-performance, schema-free document-oriented database.
Mar 03 09:50:01 p0 systemd[1]: Unit mongod.service entered failed state.
Mar 03 09:50:01 p0 systemd[1]: mongod.service failed.

# mongod --repair
2016-03-03T09:55:49.905+0000 [initandlisten] MongoDB starting : pid=21520 port=27017 dbpath=/data/db 64-bit host=comp-sscm-p0
2016-03-03T09:55:49.906+0000 [initandlisten] db version v2.6.11
2016-03-03T09:55:49.906+0000 [initandlisten] git version: nogitversion
2016-03-03T09:55:49.906+0000 [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-03-03T09:55:49.906+0000 [initandlisten] build info: Linux buildvm-08.phx2.fedoraproject.org 4.1.4-100.fc21.x86_64 #1 SMP Tue Aug 4 03:25:05 UTC 2015 x86_64 BOOST_LIB_VERSION=1_53
2016-03-03T09:55:49.906+0000 [initandlisten] allocator: tcmalloc
2016-03-03T09:55:49.906+0000 [initandlisten] options: { repair: true }
2016-03-03T09:55:49.906+0000 [initandlisten] exception in initAndListen: 10296
*********************************************************************
 ERROR: dbpath (/data/db) does not exist.
 Create this directory or give existing directory in --dbpath.
 See http://dochub.mongodb.org/core/startingandstoppingmongo
*********************************************************************
, terminating
2016-03-03T09:55:49.906+0000 [initandlisten] dbexit:
2016-03-03T09:55:49.906+0000 [initandlisten] shutdown: going to close listening sockets...
2016-03-03T09:55:49.906+0000 [initandlisten] shutdown: going to flush diaglog...
2016-03-03T09:55:49.906+0000 [initandlisten] shutdown: going to close sockets...
2016-03-03T09:55:49.906+0000 [initandlisten] shutdown: waiting for fs preallocator...
2016-03-03T09:55:49.906+0000 [initandlisten] shutdown: closing all files...
2016-03-03T09:55:49.906+0000 [initandlisten] closeAllFiles() finished
2016-03-03T09:55:49.906+0000 [initandlisten] dbexit: really exiting now

# mkdir /data
# ln -s /var/lib/mongodb /data/db

or

# mkdir -p /data/db
# chown mongodb:mongodb /data/db

# service mongod start
Redirecting to /bin/systemctl start  mongod.service
# systemctl status mongod.service
● mongod.service - High-performance, schema-free document-oriented database
   Loaded: loaded (/usr/lib/systemd/system/mongod.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-03-03 09:58:15 GMT; 55s ago
  Process: 21863 ExecStart=/usr/bin/mongod $OPTIONS run (code=exited, status=0/SUCCESS)
 Main PID: 21865 (mongod)
   CGroup: /system.slice/mongod.service
           └─21865 /usr/bin/mongod --quiet -f /etc/mongod.conf run

Mar 03 09:58:15 p0 systemd[1]: Starting High-performance, schema-free document-oriented database...
Mar 03 09:58:15 p0 mongod[21863]: about to fork child process, waiting until server is ready for connections.
Mar 03 09:58:15 p0 mongod[21863]: forked process: 21865
Mar 03 09:58:15 p0 systemd[1]: Started High-performance, schema-free document-oriented database.
Mar 03 09:58:58 p0 systemd[1]: Started High-performance, schema-free document-oriented database.

Add entry in /etc/nsswitch - Solaris 11

To add DNS to /etc/nsswitch.conf

svccfg -s name-service/switch setprop config/ipnodes = astring: '("files dns")'
svccfg -s name-service/switch setprop config/host = astring: '("files dns")'
svccfg -s name-service/switch listprop config
svcadm refresh name-service/switch

Regex with awk

To use a regular expression in awk use the operator ~ and //, e.g/:

cat home-dir.txt | awk '{if ($3 !~ /^[0-9]/) print $0;}'

to print any line that does not have a number in column 3

RedHat 7 - Software failed to update

If you have issues with yum updating your OS or packages, with this error:

Error downloading packages:
  lsscsi-0.27-3.el7.x86_64: [Errno 256] No more mirrors to try.

Mount the DVD-ROM with the RedHat EL 7 source disk:

# mount /dev/cdrom /cdrom

Make sure your yum repo has this file: cdrom.repo:

[CDROM]
name=CDROM
baseurl=file:///cdrom
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

Make sure you are joined to the RHN with a valid subscription:

And if you have this error: rhn_register fails with "ERROR: can not find RHNS CA file:/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT", fix this using the instructions here: https://access.redhat.com/solutions/26435?tour=6 correcting these lines in /etc/sysconfig/rhn/up2date:

serverURL=https://xmlrpc.rhn.redhat.com/XMLRPC
sslCACert=/usr/share/rhn/RHNS-CA-CERT

Install .net Framework 3.5 on Windows Server 2012

Start up a PowerShell as Administrator and run this:

> Install-WindowsFeature -name NET-Framework-Core -source J:\sources\sxs

Where J is your install media drive letter.

Unknown filesystem type 'LVM2_member' - Debian

So you are working with LVM volumes specifically with LVM volumes that contain complete filesystems - i.e. virtual hard disks - and you are getting this 'unknown filesystem type 'LVM2_member' when you are trying to mount the volume. This error is encountered because this partition contain multiple LVM parts (i.e. members). The way you would mount these volumes is to extract this partition, mount it as a loop device and then use pvs/lvs and vgchange command to activate the LVM and mount. Consider my LVM volume that exist on a Xen/Ganeti host:

# gnt-instance info migrate.isys

[cut]

  Disk template: plain
  Disks:
    - disk 0: lvm, size 10.0G
      access mode: rw
      logical_id:  volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data
      on primary:  /dev/volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data (254:66)
[cut]

Use kpartx to separate the partitions of the volume:

# kpartx -a -p- /dev/volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data
# ls -la /dev/mapper/volg1-50c29778*
lrwxrwxrwx 1 root root 8 Nov 30 14:46 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data -> ../dm-66
lrwxrwxrwx 1 root root 8 Nov 30 15:30 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-1 -> ../dm-79
lrwxrwxrwx 1 root root 8 Nov 30 15:30 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-2 -> ../dm-80

Let's mount the first partition: /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-1

# mount /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-1 /mnt

# ls -la /mnt
total 34110
dr-xr-xr-x.  5 root root     1024 Nov 23 13:28 .
drwxr-xr-x  23 root root     4096 May 11  2015 ..
-rw-r--r--.  1 root root   107134 Jul 23 17:13 config-2.6.32-573.el6.x86_64
drwxr-xr-x.  3 root root     1024 Nov 23 13:27 efi
drwxr-xr-x.  2 root root     1024 Nov 23 13:29 grub
-rw-------.  1 root root 27635640 Nov 23 13:29 initramfs-2.6.32-573.el6.x86_64.img
drwx------.  2 root root    12288 Nov 23 13:25 lost+found
-rw-r--r--.  1 root root   205998 Jul 23 17:14 symvers-2.6.32-573.el6.x86_64.gz
-rw-r--r--.  1 root root  2585052 Jul 23 17:13 System.map-2.6.32-573.el6.x86_64
-rwxr-xr-x.  1 root root  4220560 Jul 23 17:13 vmlinuz-2.6.32-573.el6.x86_64
-rw-r--r--.  1 root root      166 Jul 23 17:13 .vmlinuz-2.6.32-573.el6.x86_64.hmac

Looks like it's the /boot partition. Fine, the rest of my disk is on the second partition, so let's mount that:

# mount /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-2 /mnt
mount: unknown filesystem type 'LVM2_member'

Uh-oh, can not mount: unknown filesystem type 'LVM2_member'. So the second partition must be made up of more than one LVM members. Lets make an image of this and mount it as a loop back filesystem to see what is inside.

# dd if=/dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data-2 of=/var/tmp/partition2.dd bs=1M
10039+0 records in
10039+0 records out
10526654464 bytes (11 GB) copied, 174.754 s, 60.2 MB/s

# file partition2.dd
partition2.dd: LVM2 PV (Linux Logical Volume Manager), UUID: NydiFK-CaAX-PWcz-I0pY-EpAk-4ViC-pIGpf0, size: 10526654464
# losetup /dev/loop0 partition2.dd
# pvs
  PV         VG          Fmt  Attr PSize  PFree
  /dev/loop0 volg0       lvm2 a--   9.78g 992.00m
  /dev/sda5  SYSTEM      lvm2 a--  79.76g  55.55g
  /dev/sdb1  volg1       lvm2 a--   1.97t   1.96t
# lvs
  LV   VG          Attr     LSize  Pool Origin Data%  Move Log Copy%  Convert
  HOME SYSTEM -wi-ao--  3.72g
  ROOT SYSTEM -wi-ao--  7.45g
  SWAP SYSTEM -wi-ao--  3.72g
  TMP  SYSTEM -wi-ao--  1.86g
  VAR  SYSTEM -wi-ao--  7.45g
  root volg0       -wi-----  7.81g
  swap volg0       -wi-----  1.00g

Hey look! It's made up of root and a swap member! Now if you want to mount these, you can now run the vgchange command and activate them:

# vgchange -ay volg0
  2 logical volume(s) in volume group "volg0" now active

# lvdisplay

[snip]

  --- Logical volume ---
  LV Path                /dev/volg0/root
  LV Name                root
  VG Name                volg0
  LV UUID                dAoIl6-WR4G-Z0d1-9Cgg-hevk-oSaf-7TTeGv
  LV Write Access        read/write
  LV Creation host, time migrate.isys.bris.ac.uk, 2015-11-23 13:25:26 +0000
  LV Status              available
  # open                 0
  LV Size                7.81 GiB
  Current LE             250
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:5

  --- Logical volume ---
  LV Path                /dev/volg0/swap
  LV Name                swap
  VG Name                volg0
  LV UUID                L0haeP-jWyr-Gij9-sAhA-8mk6-VGS0-M42Gyl
  LV Write Access        read/write
  LV Creation host, time migrate.isys.bris.ac.uk, 2015-11-23 13:25:30 +0000
  LV Status              available
  # open                 0
  LV Size                1.00 GiB
  Current LE             32
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:6

# mount /dev/volg0/root /mnt
# ls /mnt
bin  boot  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  proc  root  sbin  selinux  srv  sys  tmp  usr  var

And there you are, you have access to your root data! Copy it, move it, do anything you want, then perform the clean up-reverse actions:

# umount /mnt
# vgchange -an volg0
  0 logical volume(s) in volume group "volg0" now active
# losetup -d /dev/loop0
# kpartx -d -p- /dev/volg1/50c29778-490a-4686-8a42-569d8b55f40c.disk0_data
# ls -la /dev/mapper/volg1-50c29778*
lrwxrwxrwx 1 root root 8 Nov 30 14:46 /dev/mapper/volg1-50c29778--490a--4686--8a42--569d8b55f40c.disk0_data -> ../dm-66