Update, compare kernel versions - reboot within 6hours period - CentOS

#!/bin/sh
host=`hostname -f`
date=`date`

# run updates to get latest software and kernel
yum -y update
maxdelay=$((6*60))
rdelay=$(($RANDOM%maxdelay))
sdelay=$(($rdelay*60))
kernel=`uname -r`
latest=`rpm -q kernel --queryformat '%{installtime} %{version}-%{release}.%{arch}\n' | sort -n -k1 | tail -1 | cut -d ' ' -f 2`
sdate=`date`

if [ "$kernel" = "$latest" ];
then
        echo "No need to reboot..."
        echo "You are running the latest kernel: $kernel"
else
        echo "There is a new kernel. You need a reboot..."
        echo "Current kernel: $kernel"
        echo "The latest kernel: $latest"
        echo "$sdate\nCurrent kernel: $kernel\nLatest kernel: $latest"
        echo "Sleeping for $sdelay seconds..."
        sleep $sdelay
        echo "Waking up and rebooting..."
        /sbin/reboot
fi

Install GRUB loader when Debian fails to boot

mount /dev/sdXY /mnt

mount --bind /dev /mnt/dev &&
mount --bind /dev/pts /mnt/dev/pts &&
mount --bind /proc /mnt/proc &&
mount --bind /sys /mnt/sys

chroot /mnt

grub-install /dev/sdX
grub-install --recheck /dev/sdX

Disable service start up at re/boot - Debian 6

The following is the chkconfig equivalents on Debian 6:

update-rc.d  <service> defaults
update-rc.d  <service> start 20 3 4 5
update-rc.d -f <service>  remove

Part of the man page:

UPDATE-RC.D(8)                                   sysv-rc                                   UPDATE-RC.D(8)

NAME
       update-rc.d - install and remove System-V style init script links

SYNOPSIS
       update-rc.d [-n] [-f] name remove

       update-rc.d [-n] name defaults [NN | SS KK]

       update-rc.d  [-n]  name  start|stop  NN  runlevel  [runlevel]...   .  start|stop NN runlevel [run‐
              level]...  . ...

       update-rc.d [-n] name disable|enable [ S|2|3|4|5 ]

DESCRIPTION
       update-rc.d updates the System V style init script links /etc/rcrunlevel.d/NNname whose target  is
       the script /etc/init.d/name.  These links are run by init when it changes runlevels; they are gen‐
       erally used to start and stop system services such as daemons.  runlevel is one of  the  runlevels
       supported  by  init,  namely, 0123456789S, and NN is the two-digit sequence number that determines
       where in the sequence init will run the scripts.

       This manpage documents only the usage and behaviour of update-rc.d.  For a discussion of the  Sys‐
       tem V style init script arrangements please see init(8) and the Debian Policy Manual.

CentOS 5 EOL - Yum repo issue

Now that CentOS 5 has gone end of life, there's no update repos left to use:

-bash-3.2# cat /etc/redhat-release
CentOS release 5.11 (Final)
-bash-3.2# yum update
Loaded plugins: downloadonly, fastestmirror
Determining fastest mirrors
YumRepo Error: All mirror URLs are not using ftp, http[s] or file.
 Eg. Invalid release/
removing mirrorlist with no valid mirrors: /var/cache/yum/base/mirrorlist.txt
Error: Cannot find a valid baseurl for repo: base

All repo contents have been moved to http://vault.centos.org so to fix the problem, the other repos must be removed:

-bash-3.2# ls /etc/yum.repos.d/CentOS-
CentOS-Base.repo       CentOS-Media.repo      CentOS-Vault.repo
CentOS-Debuginfo.repo  CentOS-Sources.repo    CentOS-fasttrack.repo
-bash-3.2# ls /etc/yum.repos.d/CentOS-

Leaving only CentOS-Vault.repo:

-bash-3.2# yum update
Loaded plugins: downloadonly, fastestmirror
Determining fastest mirrors
Setting up Update Process
No Packages marked for Update
-bash-3.2#

Boot Linux OS into shell to change root password

Get access to the console. At the grub boot options edit boot line which has "kernel" string and append:

init=/bin/sh

Boot this and the Linux OS will boot to a shell. Mount / as rewrite:

mount -o remount,rw /

Change the root password:

passwd

Reboot Linux OS:

reboot

Fix Expired Certificate Issue on Ganeti cluster

When the certificate expires on a ganeti cluster you will get these error messages when starting the ganeti service:

(0) root@server ~
# gnt-instance list
Cannot communicate with the master daemon.
Is it running and listening for connections?
(1) root@server ~
# gnt-cluster getmaster
sever.gnt6.fog.priv
(0) root@server ~
# ls /var/lib/ganeti/ssconf_master_node
/var/lib/ganeti/ssconf_master_node
(0) root@server ~
# cat /var/lib/ganeti/ssconf_master_node
sever.gnt6.fog.priv
(0) root@server ~
# /etc/init.d/ganeti start
Starting Ganeti cluster:ganeti-noded...done.
ganeti-masterd...ERROR:root:RPC error in master_info from node X: Error 60: server certificate verification failed. CAfile: /var/lib/ganeti/server.pem CRLfile: none
ERROR:root:RPC error in master_info from node X: Error 60: server certificate verification failed. CAfile: /var/lib/ganeti/server.pem CRLfile: none
ERROR:root:RPC error in master_info from node Y: Error 60: server certificate verification failed. CAfile: /var/lib/ganeti/server.pem CRLfile: none
ERROR:root:RPC error in master_info from node Z: Error 60: server certificate verification failed. CAfile: /var/lib/ganeti/server.pem CRLfile: none
CRITICAL:root:Cluster inconsistent, most of the nodes didn't answer after multiple retries. Aborting startup
CRITICAL:root:Use the --no-voting option if you understand what effects it has on the cluster state
failed (exit code 1).
ganeti-rapi...done.
ganeti-confd...done.
(0) root@sever ~
# gnt-instance list
Cannot communicate with the master daemon.
Is it running and listening for connections?
(1) root@server ~

Checking the certificate, you see that it has expired. To solve this issue, on the current (but broken) master, create a new certificate:

# cp /var/lib/ganeti/server.pem ~/expired.server.pem
# openssl req -new -newkey rsa:1024 -days 1825 -nodes -x509 -keyout /var/lib/ganeti/server.pem -out /var/lib/ganeti/server.pem -batch &&
# chmod 0400 /var/lib/ganeti/server.pem
Then run this script to copy this new certificate to all nodes in the cluster:
#!/bin/sh
for i in X Y Z
do
       ssh $i "cp /var/lib/ganeti/server.pem ~/"
       scp newserver.pem $i:/var/lib/ganeti/server.pem
       ssh $i "chmod 0400 /var/lib/ganeti/server.pem"
       ssh $i "/etc/init.d/ganeti restart"
done
/etc/init.d/ganeti restart

Note it will restart the ganeti service on all the non-master nodes before restarting the service on the master node.

Examine Certificate for expiration date

(0) root@server /var/log/ganeti
# openssl x509 -in /var/lib/ganeti/server.pem -text -noout
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number: 1 (0x1)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: CN=ganeti.example.com
        Validity
            Not Before: Mar 15 19:33:15 2012 GMT
            Not After : Mar 14 19:33:15 2017 GMT
        Subject: CN=ganeti.example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (2048 bit)
                Modulus (2048 bit):
                    00:ca:0a:cc:87:15:e5:00:7f:b0:00:21:31:3e:52:
                    09:b6:35:d3:7b:b5:60:c0:4a:e9:9c:ad:0f:cc:d8:
                    f8:2e:02:d3:fb:11:c1:77:b4:cf:1a:5a:6d:7f:7c:
                    a8:49:20:fe:df:bc:fa:94:55:a6:26:a6:9e:54:b8:
                    96:f7:aa:8d:20:a9:6f:25:64:f6:1c:89:27:c3:5c:
                    94:48:83:94:4f:2f:1b:b2:a3:ae:7f:79:d7:dc:e9:
                    29:c6:d9:88:63:8a:20:4d:03:48:7f:1e:1e:94:9c:
                    01:76:96:33:c4:46:f3:74:62:b5:af:fc:4a:99:e5:
                    b5:6f:b7:52:d5:6b:f8:a4:b0:40:7b:11:3f:6c:8d:
                    c5:ef:9e:83:02:58:10:8e:fd:03:75:4f:3e:df:95:
                    71:59:80:e8:b3:01:73:e1:a3:af:20:2c:3d:71:9b:
                    13:92:72:9c:98:ac:d4:f5:3a:f7:27:49:0c:e1:8d:
                    ed:65:18:6f:52:08:ec:11:e8:e6:c9:e6:a0:a2:3d:
                    d4:2a:3b:09:ab:09:8f:21:43:d6:95:17:55:1d:3b:
                    06:c9:e2:76:56:8f:61:bd:1f:52:96:b3:70:93:61:
                    62:ba:52:74:ef:78:51:1f:20:64:a8:5d:d1:fa:d7:
                    a6:87:9d:d0:0b:eb:19:39:ca:ed:c5:65:a1:c3:b7:
                    ac:3d
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha1WithRSAEncryption
        01:3c:b6:0f:a6:90:92:5c:4b:07:c0:16:85:a7:11:e1:fd:22:
        42:df:69:d1:be:e8:19:68:f7:40:85:af:e1:c3:30:ee:ea:7b:
        c0:ea:bc:2c:23:a2:c5:2f:65:11:d1:62:8f:da:d8:d2:01:9f:
        f8:e6:47:32:cf:e5:d1:7e:73:0d:42:20:43:19:e0:9c:3a:d7:
        a2:f9:43:b8:ca:f0:cb:2d:ae:20:d3:e7:36:c4:e5:b8:ba:c5:
        4b:96:3d:67:6a:9b:69:32:50:ba:d4:23:54:f9:ec:a5:02:25:
        29:1d:4f:b5:3f:46:e1:5f:55:0e:a5:ef:5f:dd:ca:eb:c6:8e:
        63:44:18:5c:40:a8:8d:4e:28:b2:f8:52:09:39:bd:c0:bb:80:
        c9:58:43:3d:85:2c:35:7f:da:1e:cf:ce:8f:d2:43:fd:09:fe:
        ef:0f:09:d8:a9:f9:64:d5:2d:a2:15:25:fe:e7:30:a9:18:ab:
        68:83:d0:df:1e:41:1c:3e:a6:10:f0:26:59:7b:d0:16:02:74:
        e8:f7:69:e6:64:be:50:a8:92:de:82:4a:22:f7:f2:c3:d8:4b:
        ad:2a:12:13:bb:0a:ff:fa:0c:1d:eb:f6:f3:e2:63:26:1e:01:
        a6:3b:55:65:5c:ec:11:aa:99:67:7e:95:a8:5f:73:c6:fa:7b:
        5c:4e:2f:21
(0) root@server /var/log/ganeti
#