Setting up OpenVPN on Cent OS

It has been a very log time since I’ve posted anything and thought it was about time I did. So here goes…

This article will briefly walk through the steps to setup an OpenVPN service running on a recently patched Cent OS 7.9 server and using a Fedora based OpenVPN client, where we look at both the command line configuration and Network Manager. The final step will also be to look at an Android based client.

The configuration may need some tweaking for Windows and Mac devices, sadly I don’t have any to test.

Pre-Requsities

Update base Server OS & Install packages

The first step will be to ensure the server is patched up to the latest available baseline and then we will install openvpn and easy-rsa.

For the post, I’m going to assume that easy-rsa is installed on to the OpenVPN server, HOWEVER, in a production environment you should really consider running easy-rsa on a completely seperate machine, one that can be isolated of from the network, as it will hold the private key for your certificate authority.

$ sudo yum update -y

Install OpenVPN on the server which will become your VPN server (sounds obvious I know).

$ sudo yum install -y openvpn easy-rsa

Install easy-rsa (preferrably) on a seperate machine

$ sudo yum install -y easy-rsa

Setup easy-rsa

On the server with easy-rsa installed, run the following commands.

$ sudo mkdir -p /etc/easy-rsa/pki
$ sudo cp -r /usr/share/easy-rsa/3.0.8 /etc/easy-rsa
$ cd /etc/easy-rsa
$ sudo cp /usr/share/doc/easy-rsa-3.0.8/vars.example ./vars

Now we have the base directory structure, we need to prepare the vars file which will be used to pre-populate answers to some of our questions. Uncomment the following lines in your vars file and update with the relevant information for you. For example in my lab I used;

set_var EASYRSA_REQ_COUNTRY     "GB"
set_var EASYRSA_REQ_PROVINCE    "Greater London"
set_var EASYRSA_REQ_CITY        "London"
set_var EASYRSA_REQ_ORG         "World Domination Co"
set_var EASYRSA_REQ_EMAIL       "world-domination-team@lab.tobyheywood.com"
set_var EASYRSA_REQ_OU          "Security Department"

Time to build your CA (Certificate Authority)

$ /etc/easy-rsa
$ sudo ./easyrsa clean-all
$ sudo ./easyrsa build-ca

The above commands will ensure you start with a clean slate. The option “clean-all” removes all previously generated keys and “build-ca”, well… It builds the CA using the above variables.

Create the server private key and certificate using your newly created CA

$ ./easyrsa build-server-full vpn.lab.tobyheywood.com

Stating the obvious – You will need to change vpn.lab.tobyheywood.com to the fully qualified name of your server. Well, actually it doesn’t have to be the FQDN. Many examples show this as being set to just “server”, but as we will cover later, there are reasons you may not want to do that.

When you run the above command you will be prompted to enter a passphrase to secure the servers private key. You don’t have to set a passphrase. If you don’t want to set a passphrase on your servers private key, you can add “nopass” to the command line above.

Things to consider if you do set a passphrase (the more secure option);

  • You will need to run systemd-tty-ask-password-agent which will store your passphrase in memory
  • You will need to enter this password EVERY time you start the OpenVPN server service
  • After every reboot both points above will need to be performed
  • If you are setting up the OpenVPN server to reduce the ports exposed to the internet (which is a good idea) and you disable SSH access via your public facing interface (for example its a server in a colo environment), you will definitely need another method of connecting into your server, whether that is simply that you have physical console access, out of band (OOB) management like HPE iLO or Cisco’s IMC, or even simply via another server in your network.

Reasons NOT TO set a passphrase on your server certificate (arguably a less secure option, IF your server gets compromised);

  • You don’t need to worry about how you are going to handle unexpected reboots, by not having a passphrase means that the OpenVPN service should just start
  • You don’t need to store a password in memory

The choice is your! I’m not going to make a recommendation one way or the other, as depending on the use case, the answer could be very easy be either.

Next we need to generate the Diffie-Hellman key exchange file

$ ./easyrsa gen-dh

At this point you should have configured, the CA (Certificate Authority), which in turn has created the required ca.crt and ca.key files. In addition we have also created a private key and certificate for the server itself and generated the Diffie Hellman key exchange file too.

We now need to copy the following files into the /etc/openvpn directory;

$ sudo cp pki/issued/vpn.tobyheywood.com.crt /etc/openvpn/
$ sudo cp pki/private/vpn.tobyheywood.com.key /etc/openvpn/
$ sudo cp pki/ca.crt /etc/openvpn/
$ sudo cp pki/dh.pem /etc/openvpn/
# Make sure the files are owned by root
$ sudo chown root.root /etc/openvpn/*
# Lets not forget to make sure the selinux security contexts are set properly
$ sudo restorecon /etc/openvpn

We will need to come back to easy-rsa in order to create our client certificates. I shall cover that process shortly

Server Configuration

OpenVPN Configuration

One thing I found when looking for guidance on configuring OpenVPN is that the example configurations don’t tend to provide explanations fas to why they have deviated from the sample configs provided by OpenVPN. One such example is why you should use “tls-crypt” rather than “tls-auth”. I shall cover that off in a short while with some other pertinent pieces of information.

$ sudo cp /usr/share/doc/openvpn-2.4.10/sample/sample-config-files/server.conf /etc/openvpn/

Now that you you have copied the sample server.conf file, take a look using your favourite text editor. There are many options, and masses of commented out sections (lines starting with a “;” or a “#”). I shall focus on what you will need to get started (don’t forget the man page for openvpn, as it lists every option and provides plenty of detail).

You can take the below and create your own server.conf file instead of using the sample if you prefer not to have all of the commented out options.

# The port number - this is the default
port 1194

# This can be set to udp or tcp.  Recommendation is to ensure this is set to udp.
# It provides better performance.
proto udp

# This can be set to tun or tap and controls the type of VPN tunnel used.  From the
# sample conf file;
# "dev tun" will create a routed IP tunnel,
# "dev tap" will create an ethernet tunnel.
dev tun


# The Certificate Authority certificate
ca /etc/openvpn/ca.crt
# The OpenVPN server certificate
cert /etc/openvpn/vpn.lab.tobyheywood.com.crt
# The OpenVPN server private key
key /etc/openvpn/vpn.lab.tobyheywood.com.key

# The Diffie Hellman key exchange certificate
dh dh.pem

# Used to define how the network will be divided up to the client.  For example
# older configurations would allocate a /30 to each client.  This is not
# recommended.  The recommended setting is subnet, essentially meaning the client
# is allocated a single IP within the entire subnet allocated for clients.
topology subnet

# The VPN subnet.  Using the below will result in the server being assigned
# 10.10.10.1 and client connections will start from 10.10.10.2 and go all the
# way up to 10.10.10.254
server 10.10.10.0 255.255.255.0

# This file keeps trace of the vpn ip addresses assigned to clients.
ifconfig-pool-persist ipp.txt

# These are the DNS server IPs that will be provided to clients once a VPN tunnel
# has been established.  Personally I don't go near Google's free DNS service, as 
# I'm pretty sure they will log all of the websites and services my users access, 
# but if you don't have an option...
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 7.7.7.7"

# Tell clients to route all traffic via the OpenVPN servers IP address rather than
# what is provided by DHCP via the local network
push "redirect-gateway def1 bypass-dhcp"

# Send a ping across the tunnel every 10 seconds, and assume the peer is down if no
# ping received within 120 seconds
keepalive 10 120

# See below for reason why you should use tls-crypt rather than the default
# tls-auth
tls-crypt vpn.lab.tobyheywood.com.tlsauth

# Enforce the use of a modern encryption cipher.  OpenVPN 2.4+ will auto negotiate
# this when in TLS mode.
cipher AES-256-GCM

# Control the maximum number of client connections.
max-clients 10

# Run OpenVPN as user nobody and group nobody after initial startup.
user nobody
group nobody

# These options try to avoid accessing certain resources (for example the the
# private key and tun device) after downgrading from root to nobody.
persist-key
persist-tun

# OpenVPN service status
status openvpn-status.log

# OpenVPN main log file
log         openvpn.log

# Logging verbosity - 1-9 can be set, with 9 being really, REALLY verbose, you
# you probably don't need more than 6 when debugging
verb 3


# Tell clients when the server has restarted so they can autoreconnect.
explicit-exit-notify 1

# When the certificate is generated for either the server or the client, it
# has an additional attribute added to it.  The Extended Key Usage attribute is
# used in conjunction with tls-crypt.
remote-cert-eku "TLS Web Client Authentication"

TLS crypt verses TLS auth – This is discussed here https://serverfault.com/questions/929484/openvpn-2-4-security-differences-between-tls-crypt-and-tls-auth and based on the excellent explanation I would recommend using tls-crypt rather than tls-auth s it is more secure.

remote-cert-eku – This is used to verify the certificate presented by the server/client. As mentioned above when the certificate is created it has an additional attribute added to it, this is called the Extended Key Usage, this is subtly different depending on whether the OpenVPN configuration is for a Server or a Client.

  • Server – Expects client certificates to have “TLS Web Client Authentication”
  • Client – Expects the server certificate to have “TLS Web Server Authentication”

Next we need to generate the the tls auth file.

$ openvpn --genkey --secret /etc/openvpn/vpn.lab.tobyheywood.com.tlsauth

Configuring the firewall (firewalld)

Assuming you have followed the configuration examples thus far, the next step will be to configure the firewall. There are two sides to this.

  1. The Public zone – The zone which has been configured with your public facing interface
  2. The Trusted zone – This is the zone which we need to configure to allow traffic to flow through the tunnel interface (tun0).
$ sudo firewall-cmd --permanent --zone=public --add-service openvpn
$ sudo firewall-cmd --permanent --add-masquarade
$ sudo firewall-cmd --permanent --direct --passthrough ipv4 -t nat A POSTROUTING -s 10.10.10.0/24 -o eno6 -j MASQUERADE
$ sudo firewall-cmd --permanent --zone=trusted --add-interface tun0
$ sudo firewall-cmd --reload

Note. Using the above example you will need to replace 10.10.10.0/24 with your own desired VPN subnet and also eno6 with your Internet facing (public) interface.

Configuring IP forwarding

Assuming you want to be able to connect to the Internet via your OpenVPN server, you will need to enable IPv4 forwarding. This can be achieved by running the following commands

$ sudo -i
# echo "net.ipv4.ip_forward = 1" > /etc/sysctl.conf
# exit
$ sudo systemctl restart network.service

Start and Enable openvpn@server.service

The last step is to enable the OpenVPN server so that it starts automatically after a reboot and also whilst we are at it, lets make sure it has started up without issues.

$ sudo systemctl start openvpn@server
$ sudo systemctl enable openvpn@server
$ sudo systemctl status openvpn@server
$ sudo cat /etc/openvpn/openvpn.log

Generating Client Certificates

Now that everything looks good from a server perspective we can start to focus on getting a client or two up and running. All clients require a certificate for authentication purposes. And the process is pretty simple.

All of the following steps need to be performed on the same machine which was used to create your CA (Certificate Authority). For the client certificate again you have the option to not specify a passphrase using the “nopass” parameter, however for client certificates I encourage you to use a strong passphrase.

$ ./easyrsa build-client-full myclient.lab.tobyheywood.com

Note: using Easy-RSA configuration from: /home/toby/easy-rsa/vars
Using SSL: openssl OpenSSL 1.1.1k  FIPS 25 Mar 2021
Generating a RSA private key
.......................................................+++++
.....................................................................................................+++++
writing new private key to '/home/toby/easy-rsa/pki/easy-rsa-24006.mj1k8n/tmp.HkGd2m'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
Using configuration from /home/toby/easy-rsa/pki/easy-rsa-24006.mj1k8n/tmp.9x5fSs
Enter pass phrase for /home/toby/easy-rsa/pki/private/ca.key:
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'myclient.lab.tobyheywood.com'
Certificate is to be certified until Jul 22 18:35:04 2023 GMT (825 days)

Write out database with 1 new entries
Data Base Updated

Now that you have generated the new private key and certificate for your new client, we now need to create a configuration file for the client.

$ cd
$ mkdir -p client_config_bundle/myclient
4 vim client_config_bundle/myclient/myclient.labtobyheywood.com.ovpn

Using the below as a template, and paste it into a blank text file.

tls-client
ca ca.crt
cert myclient.lab.tobyheywood.com.crt
key myclient.lab.tobyheywood.com.key

tls-crypt vpn.lab.tobyheywood.com.tlsauth
remote-cert-eku "TLS Web Server Authentication"

# Remote end point - replace 1.2.3.4 with your openvpn servers public IP
proto udp
remote 1.2.3.4 1194 udp

I would recommend the file name is based on the clients hostname and it should have the suffix .ovpn (this is beneficial if using the OpenVPN Connect client on your Android device).

$ cd /etc/openvpn/easy-rsa/   # Or where ever you have installed easy-rsa
$ cp issued/myclient.lab.tobyheywood.com.crt client_config_bundle/myclient/
$ cp private/myclient.lab.tobyheywood.com.key client_config_bundle/myclient/
$ cp ca.crt client_config_bundle/myclient/
$ sudo cp ../vpn.lab.tobyheywood.com.tlsauth client_config_bundle/myclient/
$ cp <path to client configuration file> client_config_bundle/myclient/

Configuring your clients

Client Configuration – Command line using openvpn

Above, you will have hopefully created a directory which contains all the files you need to make a successful client connection. To recap, you should have;

  • your-clients.crt
  • your-client.key
  • ca.crt
  • you-vpn-server.tlsauth
  • your-client.ovpn
$ sudo openvpn --config /path/to/your/client-config.ovpn

Because OpenVPN will create a new bridged interface, the example below will need to be run via sudo. If however you want to remove the need for running it with sudo you can by configuring Unprivileged mode (Linux only), and it is discussed here (just scroll down to the Unprivileged mode section – https://openvpn.net/community-resources/hardening-openvpn-security/

Client Configuration – Network Manager

Configuring OpenVPN via the gnome NetworkManager applet is pretty straight forward.

Note. This method does require you to store your password on the machine, which will be stored on disk. If you are concerned in anyway, this option may not be for you.

In a terminal session, type the following to install the required additional package.

$ sudo dnf install NetworkManager-openvpn
  • Press the Super button or click on top left hand corner of your screen (Gnome)
  • Type settings and select the Settings icon
  • You should see a window similar to below
  • Click the + to the right of VPN.
  • Select OpenVPN
NetworkManager VPN options.

In the Add VPN window, you will need to provide;

  • A name for the new VPN connection
  • General
    • Gateway – This is either the fully qualified domain name of your openvpn server or its IP address (For the real secret concious of you, IP address is the better way to go as there will be no DNS lookup involved)
  • Authentication
    • Type – Certificate (TLS)
    • CA Certificate – Select your ca.crt file
    • User certificate – Select your client certtificate file
    • User Private Key – Select your client private key file
    • User key password – Enter the passphrase for your private key
    • Select the Icon on the far right of the password field and select Store for user only

As mentioned above, the icon on the far right of the password field can be clicked to show different options. I have found that as my user doesn’t have sufficient access to start up network connections, without using sudo, that the only successful way to make this work is to store your password, or have no password at all.

Example OpenVPN Identity configuration screen in NetworkManager

I found a discussion here which talks about the type of error I see in journalctl, which for reference is “Failed to request VPN secrets #1: No agents were available for this request“. I suspect that if you configure OpenVPN to run as unprivileged user from the very start and you manually configure the tun/tap interface that you will probably find the password prompt will work. But I haven’t tested this.

  • Now click Advanced and select “Verify name exactly” and enter the name of your server (as you provided when you created the openvpn server certificate). This is where have the fully qualified name can be a benefit because it means something to everyone, rather than just having server which doesn’t really help anyone understand what you are connecting to.
  • Set Mode as TLS-Crypt
  • Select the tlsauth file that you generated earlier

The last step is click apply. This will return you to the

Once you have applied your configuration as above with your ca.crt, your private key and certificate, plus your tlsauth file you should now be able to start your VPN tunnel. If it doesn’t work I would suggest making sure you can connect via the client first and then compare the parameters in your ovpn file with what you have set in the Network Manager window.

Client Configuration – Android

As with the other client configurations, there isn’t really much to it, so lets get stuck in.

Pre-requisites

  • Create a new client private key and cert
  • Copy the private key and cert to a dedicated folder along with the tlsauth file and the ca.crt file
  • Create a .ovpn file for your client
  • Install the OpenVPN Connect app on your device

I use a secure cloud storage service to get all the files onto my device, but you could connect your device to your computer via USB and copy the files across.

  • Once you have all the required files stored on your device, launch the OpenVPN connect app
  • Select file
  • Navigate through the Android internal storage to where you have copied your certificates, keys and configuration file
  • Select the .ovpn file
  • Tap Import
  • Either accept the default Profile Name or change it to something more to your liking
  • If you wish you can store the Private Key password/passphrase in the app
  • Tick the Connect after import option
  • Click Add
  • You should now have a successful connection to your OpenVPN server

kernel: BUG: soft lockup – CPU#0 stuck for 67s!

Over the weekend I was confronted by the above error being repeated on the console of a VM running Oracle RDBMS.

This error occurs when there is a shortage of CPU resources.  For me the solution was a quick shut down of the VM and increasing the available CPU resources.  However there are more ways to skin this cat…

There is also a kernel parameter which can be tweaked;

kernel.softlockup_thresh=x

Were “x” is the threshold (in seconds) you want to allow the kernel to wait before decided there has been a soft lockup.

The Red Hat documentation showed a threshold of 30 seconds.  So I would recommend a bit of experimentation if you feel that 30 seconds is not high enough.  Or throw more resources at it.

Reference Material

Featured image: The Old Lockup was made available by John Powell on Flickr.

Picture of fiber connected switches and servers

Back to basics – Creating a centralised yum/dnf repository

So far in the “Back to Basics” series (if you can call it that), I’ve covered, setting up a local yum repository, creating a internal DNS server and creating a DHCP server.  Oh, and also then correcting the fact that I had missed the reverse DNS zone for my lab network!  Doh!!!  Now, none of this was by accident (accept the reverse zone mishap).

In an isolated network, access to installation media can be essential, DNS and DHCP a pretty much standard in all environments (there are some exceptions) and all are pretty much mandatory in order to get your network up and running.

The ultimate aim of this series is to end up with a server which can be used to build more servers and/or clients into the lab network that I am setting up.

Before we can reach this goal, there are a few outstanding things to tackle;

  • Making our local repository readable from within our network (this article)
  • Setting up a TFTP server and confirm it works
  • Enable PXE booting functionality via DHCPd
  • Customising our installs using Kickstart

The above will then provide a basic but functional method of deploying more servers and clients into the lab environment, across the network and removes the need for monkeying around with ISO images, USB sticks (if you were to do similar in a real network) and once tested removes the human errors that can be introduced when manually installing an OS multiple times.

So what do we need

  • Apache (a.k.a. httpd)

Installing Apache

Given that I set up a local yum repository based on the installation media, it couldn’t be simpler.

[bash][toby@rhc-server ~]$ sudo yum install httpd
Loaded plugins: fastestmirror
baselocal                                                                                                            | 3.6 kB  00:00:00     
Loading mirror speeds from cached hostfile
Resolving Dependencies
–> Running transaction check
—> Package httpd.x86_64 0:2.4.6-17.el7.centos.1 will be installed
–> Processing Dependency: httpd-tools = 2.4.6-17.el7.centos.1 for package: httpd-2.4.6-17.el7.centos.1.x86_64
–> Processing Dependency: /etc/mime.types for package: httpd-2.4.6-17.el7.centos.1.x86_64
–> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-17.el7.centos.1.x86_64
–> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-17.el7.centos.1.x86_64
–> Running transaction check
—> Package apr.x86_64 0:1.4.8-3.el7 will be installed
—> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed
—> Package httpd-tools.x86_64 0:2.4.6-17.el7.centos.1 will be installed
—> Package mailcap.noarch 0:2.1.41-2.el7 will be installed
–> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================
 Package                         Arch                       Version                                     Repository                     Size
============================================================================================================================================
Installing:
 httpd                           x86_64                     2.4.6-17.el7.centos.1                       baselocal                     2.7 M
Installing for dependencies:
 apr                             x86_64                     1.4.8-3.el7                                 baselocal                     103 k
 apr-util                        x86_64                     1.5.2-6.el7                                 baselocal                      92 k
 httpd-tools                     x86_64                     2.4.6-17.el7.centos.1                       baselocal                      77 k
 mailcap                         noarch                     2.1.41-2.el7                                baselocal                      31 k

Transaction Summary
============================================================================================================================================
Install  1 Package (+4 Dependent packages)

Total download size: 3.0 M
Installed size: 10 M
Is this ok [y/d/N]: y
Downloading packages:
——————————————————————————————————————————————–
Total                                                                                                        12 MB/s | 3.0 MB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : apr-1.4.8-3.el7.x86_64                                                                                                   1/5
  Installing : apr-util-1.5.2-6.el7.x86_64                                                                                              2/5
  Installing : httpd-tools-2.4.6-17.el7.centos.1.x86_64                                                                                 3/5
  Installing : mailcap-2.1.41-2.el7.noarch                                                                                              4/5
  Installing : httpd-2.4.6-17.el7.centos.1.x86_64                                                                                       5/5
  Verifying  : mailcap-2.1.41-2.el7.noarch                                                                                              1/5
  Verifying  : httpd-2.4.6-17.el7.centos.1.x86_64                                                                                       2/5
  Verifying  : apr-util-1.5.2-6.el7.x86_64                                                                                              3/5
  Verifying  : apr-1.4.8-3.el7.x86_64                                                                                                   4/5
  Verifying  : httpd-tools-2.4.6-17.el7.centos.1.x86_64                                                                                 5/5

Installed:
  httpd.x86_64 0:2.4.6-17.el7.centos.1                                                                                                      

Dependency Installed:
  apr.x86_64 0:1.4.8-3.el7   apr-util.x86_64 0:1.5.2-6.el7   httpd-tools.x86_64 0:2.4.6-17.el7.centos.1   mailcap.noarch 0:2.1.41-2.el7  

Complete![/bash]

Confirm file and folder permissions

[bash][toby@rhc-server ~]$ ll -Z /software/
drwxr-xr-x. root root unconfined_u:object_r:default_t:s0 centos7
[/bash]

One thing to consider here is SELinux.

Before you run for the hills screaming, just take a deep breath and embrace something that by default will make your server more secure, yes it does have a bit of a learning curve but doesn’t everything?

I’m a believer in using the tools available to ensure I end up with a secure and stable environment.  SELinux is one of those things which for many years I avoided like the plague but to be honest, that was due to me not having had the time to properly understand what it does and how it does it.

After spending some time tinkering with it, it didn’t seem half as scary.  For sure, it complicates things a little when you come to troubleshoot permission issues, but then everything is more contained.

Lets make sure that the directory containing the installation media, which is in a none standard location (as far as Apache is concerned), has the correct SELinux permissions assigned to the folder structure.  The easiest way is to copy the existing SELinux contexts from the /var/www/html and here is the command to do that;

[bash][root@rhc-server ~]# cd /var/www/
[root@rhc-server www]# ll -Z
drwxr-xr-x. root root system_u:object_r:httpd_sys_script_exec_t:s0 cgi-bin
drwxr-xr-x. root root system_u:object_r:httpd_sys_content_t:s0 html
[root@rhc-server www]# chcon -R –reference=/var/www/html/ /software/centos7
[root@rhc-server www]# ll -Z /software/
drwxr-xr-x. root root system_u:object_r:httpd_sys_content_t:s0 centos7
[/bash]

As you can see we now have the right context associated with the centos7 directory, now we need to make sure the httpd.conf file is updated to present the centos7 directory and it’s contents to the outside world.

Rather than modifying the httpd.conf file itself it is recommended that you create your own .conf files in /etc/httpd/conf.d/ and these will be loaded after the initial httpd.conf file.  I created a single test files as follows;

[bash][toby@rhc-server ~]$ cat /etc/httpd/conf.d/software.conf
Alias "/centos7" "/software/centos7"

<Directory /software/centos7>
Options +Indexes

Order allow,deny
Allow from all
Require all granted
</Directory>
[/bash]

Screenshot showing successful directory listing from client machine of centos7 media
Screenshot showing successful directory listing from client machine of centos7 media

The Alias allows you to point to a directory which is outside of Apaches’ DocumentRoot, (typically set to /var/www/html).  The Directory block, contains two things of note.  First, for testing I have added the “Options +Indexes” so that when I try to connect from a web browser on my client machine, I can confirm that I can see the contents of the repository directory.  The second chunk of config, starting “Order all,deny…” is there so that Apache will allow connections to this none standard location.

One thing I did have to do, that I haven’t stated above is allow HTTP connections through the firewall.

This was accomplished by way of a simple one liner;

[bash][toby@rhc-server ~]$ sudo firewall-cmd –zone=public –add-service=http[/bash]

Note.  To make this new firewall rule permanent you need to use the “–permanent” firewall-cmd option on the command line.  I added this afterwards once I was happy that everything was working.

Configuring a yum .repo file to access the centralised software repository

This is very similar in the steps taken when I setup the local yum repository.  The only difference this time will be that I’ll give it a more meaningful name and the file location will be a http:// address rather that a file:///.

So here is what I have put together;

[text][root@rhc-client yum.repos.d]# cat CentOS-lab-Media.repo
[th_lab_server]
baseurl=http://rhc-server.lab.tobyheywood.com/centos7/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
[/text]

And then, as the saying goes, the proof is in the pudding;

[bash][root@rhc-client yum.repos.d]# yum repolist
Loaded plugins: fastestmirror, langpacks
Repository ‘th_lab_server’ is missing name in configuration, using id
th_lab_server                                                                                                        | 3.6 kB  00:00:00     
(1/2): th_lab_server/group_gz                                                                                        | 157 kB  00:00:00     
(2/2): th_lab_server/primary_db                                                                                      | 4.9 MB  00:00:00     
Loading mirror speeds from cached hostfile
repo id                                                            repo name                                                          status
th_lab_server                                                      th_lab_server                                                      8,465
repolist: 8,465
[/bash]

Oops, now it would appear I haven’t added a name parameter in the dot repo file.  Let me correct that…

[text highlight=”3″][root@rhc-client yum.repos.d]# cat CentOS-lab-Media.repo
[th_lab_server]
name="CentOS7 Media on rhc-server.lab.tobyheywood.com"
baseurl=http://rhc-server.lab.tobyheywood.com/centos7/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
[/text]

And now if I run the command “yum repolist” again, it should return the list of enabled repositories without complaining, oh and the repo name column will also show my desired name for my network enabled repository ( a shorter name may be better);

[bash highlight=”5″][root@rhc-client yum.repos.d]# yum repolist
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
repo id                                          repo name                                                              status
th_lab_server                                    "CentOS7 Media on rhc-server.lab.tobyheywood.com"             8,465
repolist: 8,465[/bash]

And there we have it a working centralised repository, if you don’t have access to Red Hat Satellite server or if you don’t want to install the open source version Spacewalk.

I guess the final test, would be to install a couple of packages;

[bash highlight=”3,5,7,10-14,21,24″][root@rhc-client yum.repos.d]# yum install iostat
Loaded plugins: fastestmirror, langpacks
th_lab_server                                                                                                        | 3.6 kB  00:00:00
Loading mirror speeds from cached hostfile
No package iostat available.
Error: Nothing to do
[root@rhc-client yum.repos.d]# yum whatprovides iostat
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
th_lab_server/filelists_db                                                                                           | 5.8 MB  00:00:00
sysstat-10.1.5-4.el7.x86_64 : Collection of performance monitoring tools for Linux
Repo        : th_lab_server
Matched from:
Filename    : /usr/bin/iostat

sysstat-10.1.5-4.el7.x86_64 : Collection of performance monitoring tools for Linux
Repo        : @anaconda
Matched from:
Filename    : /usr/bin/iostat

[root@rhc-client yum.repos.d]# yum install sysstat -y
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
Package sysstat-10.1.5-4.el7.x86_64 already installed and latest version
Nothing to do
[/bash]

Ah, I guess a demo is never meant to go really smoothly, but this is probably better as it demonstrates some awesome functionality that yum has.

Line 3, shows that it has found my networked repo, line 5 shows that there is no such package in the repo, line 7 highlights a way to find the right package for the command you want to run, lines 10 through 14 show conclusively that it has connected across the network to the repo, and has found a suitable package called sysstat. In line 21, I’m trying to install the package only to be told in line 24 that it’s already installed.

The really keen eyed of you may have also spotted the @anaconda repo, this should have rung an alarm bell in my head to say, hey!  What are you doing?  Its already installed!

Useful link – RPM DB Recovery

Every now and then, we find ourselves in a bit of a predicament.  In this instance whilst performing an upgrade on a server, things just weren’t going well and it appeared we had some corruption in the RPM database on one of our servers.

We were seeing segmentation faults when trying to use “rpm”.

The following page on the rpm.org website proved very useful, in getting things back up and running swiftly;

http://www.rpm.org/wiki/Docs/RpmRecovery