Recently, I’ve been working on deploying a clustered Instant Messaging (IM) chat service in my lab and after setting up the clustering by way of the Hazelcast plugin, I found that I was have some rather strange errors being written into the log files which suggested that the server to server connectivity was not being successfully initiated.
Here is a snippet from the log file;
2016.09.14 17:51:13 WARN [Server SR - 16593225]: org.jivesoftware.openfire.net.SocketReader - Closing session due to incorrect hostname in stream header. Host: of1.lab.tobyheywood.com. Connection: org.jivesoftware.openfire.net.SocketConnection@c53ac0 socket: Socket[addr=/192.168.1.11,port=44042,localport=5269] session: null
2016.09.14 17:51:13 WARN [Server SR - 3158473]: org.jivesoftware.openfire.net.SocketReader - Closing session due to incorrect hostname in stream header. Host: of1.lab.tobyheywood.com. Connection: org.jivesoftware.openfire.net.SocketConnection@1f8e35b socket: Socket[addr=/192.168.1.11,port=44043,localport=5269] session: null
2016.09.14 17:51:13 WARN [pool-10-thread-3]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: openfire.lab.tobyheywood.com to RS at: of1.lab.tobyheywood.com (port: 5269)] - Unable to create a new outgoing session
2016.09.14 17:51:13 WARN [pool-10-thread-3]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: openfire.lab.tobyheywood.com to of1.lab.tobyheywood.com] - Unable to create a new session: Dialback (as a fallback) failed.
2016.09.14 17:51:13 WARN [pool-10-thread-3]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'openfire.lab.tobyheywood.com' to remote domain: 'of1.lab.tobyheywood.com'] - Unable to authenticate: Fail to create new session.
Now as part of my investigation into the issue I noticed that the servers were not listing on the server to server port (TCP port 5269). Which in all honesty confused me even more.
A bit of Googling later (admit it, we all do it sometimes), and I had found the solution.
From the Openfire Web UI, Navigate to the following location and set the STARTTLS Policy to “Required“.
Server > Server Settings > Server to Server > STARTTLS Policy = Required.
Do this for both (or all) nodes and restart the services. You should find that things are looking happier.
Yesterday I had an interesting issue, where one of the colo’d servers I manage, became almost unresponsive. After what felt like an age I was finally logged on to the failing box in question. It quickly became apparent that the ClamAV application clamscan was being spawned multiple times and consuming all available memory and causing the server to swap until it ran out of swap (in fairness the server is lacking in the memory department).
I initially thought it might be related to the AV scanning of emails, but the server wasn’t handling a lot of email at the time. I then started to dig deeper and as the clamscan processes were owned by apache I started to question whether or not spamassassin was doing some weird and wonderful scanning by way of a clamav plugin. Not the case. In the end it turns out that it was being triggered by the ownCloud 9.x installation on the same server. At some point, Antivirus integration was enabled within ownCloud (I believe prior to ClamAV being installed). It also happened that there was about 1.6GB of data being uploaded to that server which was triggering all of the clamscan processes.
Clamscan is (by ownClouds own admission) not the best option on grounds of performance and reliability, which does beg the question; if it’s not the most suitable way to use ClamAV in an ownCloud installation, why default to that setting???
For me I uninstalled ClamAV which in turn prevented the clamscan app from being executed and I was then able to access ownCloud again to make the necessary changes to the settings so that clamscan was not used.
Changing the method from Executable to one of the Daemon methods is advisable as a minimum. After changing to the daemon (socket) method it was a more controlled use of the av scanning and I haven’t so far seen any ill effect.
Just a thought… Following the EU referendum and the subsequent rift which was highlighted by the very close vote, plus the protests that have taken place since, but just maybe, the United Kingdom is not necessarily as united as we may hope.
Hopefully the new government led by prime minister Theresa May will pull it out of the bag and be able to rebuild and reunify the countries that make up the UK, only time will tell!
Currently the UK enjoys the use of the .uk domain space. If the UK were to fracture further, at what point will/could .uk become surplus to requirements (if the worst happened)? I guess the simply answer is when we are simply no longer united and the government has agreed to relinquish further control.
Would the .gb domain name need to be re-instated, which though still online, no one can register new domains. It is managed by an organisation called JISC which provides much of the academic IT infrastructure used by UK schools, colleges and maybe even university (I didn’t do that much research in all honesty).
I wounder if the there are already plans in the pipeline? Also how far along are they?
Just think though, the opportunities are there to commercialise the .gb domain. Had it been available, maybe Great Britain’s Athletes would have bragged about the whopping medal collect following Rio 2016, they could have had the domain team.gb?
Over the weekend I was confronted by the above error being repeated on the console of a VM running Oracle RDBMS.
This error occurs when there is a shortage of CPU resources. For me the solution was a quick shut down of the VM and increasing the available CPU resources. However there are more ways to skin this cat…
There is also a kernel parameter which can be tweaked;
Were “x” is the threshold (in seconds) you want to allow the kernel to wait before decided there has been a soft lockup.
The Red Hat documentation showed a threshold of 30 seconds. So I would recommend a bit of experimentation if you feel that 30 seconds is not high enough. Or throw more resources at it.
When it comes to setting up Spacewalk to provide and meet you organisations package management and provisioning needs, there is more to it than simply installing the Spacewalk and then clicking Provision! There is a list of hoops to jump through before you can get up and running. This post aims to tackle the common setup tasks through to your first client registration, but specifically with respect to installing it on a system running CentOS 7. But lets not get ahead of ourselves. There is a lot to do, so lets get cracking!
I am assuming here that you have managed to install Spacewalk and are now looking for the next steps starting at creating the administrative user. If not may I suggest taking a peek here as I have provided a very rough guide to do this. I am also basing a lot of these steps on the HowTo published on the CentOS wiki, though that was for release 5 of CentOS, not 7. So I will try to fill in gaps where required.
Initial configuration of Spacewalk
OK, you have at this point, hopefully got Spacewalk/Satellite server installed. The first thing to do at this point is to login to the web GUI. Well, when I say login, I mean create the admin account. Best to do this right away before someone has the opportunity to takeover your nice new Spacewalk/Red Hat Satellite server. You access the GUI by typing in the FQDN of the spacewalk server and it will redirect you to the Create Spacewalk Administrator. You should see a screen much like the following;
Just enter a few essential details and away you go. OK you won’t get too far yet, but keep reading!
Upon clicking the “Create Login” button, you should see the normal dashboard screen that is displayed when logging into Spacewalk (and Red Hat Satellite) for the first time. With one exception. You should also have a banner across the top with the following wording;
You have created your first user for the Spacewalk Service. Additional configuration should be finalized by Click here
Make sure you click the “Click here” link and make sure you complete the rest of the steps.
I would advise you check and double check the General configuration tab, specifically the Spacewalk hostname, this should ideally match the FQDN of your satellite server. And if you haven’t specified a name which is resolvable via DNS you will likely find that things don’t run exactly as they should.
The Certificate tab will be of interest if you are minting your own SSL certificates or wish to use a commercially generated cert. The Bootstrap script tab is where you define settings relating to how clients connect and the associated security around those connections.
The Organizations tab (which in my opinion should read Organisations (because that’s how you spell it) is where you can define how your organisation looks, you can define multiple activation keys for different parts of your organisation, manage subscriptions and users. To name but a few of the things you can do.
Restart tab, er, do I really need to suggest what this does??? And finally the Cobbler tab. From here you can kick of a synchronisation between Spacewalk and Cobblerd. I recommend clicking it now to make sure the integration between the two applications is working. I would also suggest you double check the cobbler log file located at /var/log/cobbler/cobbler.log for any signs of problems. Here’s a sample output;
In order to register systems against your newly installed Spacewalk server, you must have a activation key defined. This is not done automatically, and therefore we shall tackle it now. Navigate to Systems > Activation Keys.
Initially you should see a message stating;
You do not currently have a universal default activation key set. To set a key as the universal default, please visit the details page of that key and check off the 'Universal Default?' checkbox.
Click + Create Key in the top right hand corner of the Activation Keys screen. You will need to add the following details;
A Key (I would advise putting something meaningful in here, rather than allowing a key to be auto-generated.
Leave the Usage field blank
Leave the Base Channels as default (Spacewalk Default)
Add-on Entitlements, I have selected only Provisioning (it can be changed later)
I also ticked the Universal Default as I do not want to restrict its use
After the default key has been created, the screen looks like this;
Creating your first package repository (and channel)
I will be focusing on CentOS 7 here, but Satellite is capable of providing a centralised repository for other RPM based distributions.
CentOS 7 Base Repository
We will assume you may at some point want to build further servers using the base OS RPMs. The first thing you need to do is find a local mirror site which you can base your repository on. CentOS provide a lovely page on their web site – https://www.centos.org/download/mirrors/, which details, (by country) where you can download the packages from. In my case I searched the page for United Kingdom and picked one from the list.
Lets get on with the job at hand, and create a repository. Click Channels > Manage Software Channels > Manage Repositories. And then click Create Repository. You will then see a screen, not too dissimilar to the one below;
Once you have defined the repository label and URL (this is the source url which Spacewalk will be using to obtain the packages from). I have defined the SSL cert that was generated during installation.
You will now need to create the Channel that will be associated to this channel. Click Manage Software Channels from the left hand menu and then click on Create Channel. You will (once the page has loaded) be given a few ground rules regarding naming conventions and then the opportunity to create your new channel. The long and short of it is this;
Channel Name and Channel Label are required (hence the red asterisk)
must be between 6 and 256 characters in length
must begin with a letter
may contain spaces, parentheses () and forward slashes /.
Channel Label must;
be no longer than 128 characters
start with a letter or digit
Must be lowercase (no exceptions)
May contain hyphens, periods, underscores and numerals
Some other options on the screen also include controlling access to the repository (i.e. is it private and only accessible to your Spacewalk organisation, or is it public), also you can define GPG security settings for signed packages.
The last step is to marry the repository and channel together. This is achieved by going to the Repositories tab, and selecting the repository from the list of available repositories. In my case it is just one.
The last step, will be kick off a synchronisation of the repository. Now there are two ways to do this; 1) By click the Sync tab, then ticking the “Create kickstartable tree” option and then clicking Sync Now. Or 2) run the following command from the cli.
Now sit back and watch/wait. For the current 7.2 repo, the base number of packages is just over 9000, so depending on your connection to the internet, you could find this to be a quick process or quite slow (also very dependent upon the mirror you have selected). Another option which I haven’t don’t but I believe it would work, is to use a copy of the installation media. If you try that option, let me know how you get on 🙂
Registering your first client
First, you will need to make sure you have the required packages on the client to be register. In my case, I had used a minimal install and as such I was missing the required packages. Easily rectified;
[root@rhc-client ~]# yum install rhn-setup
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
--> Running transaction check
---> Package rhn-setup.noarch 0:2.0.2-6.el7 will be installed
--> Processing Dependency: rhn-client-tools = 2.0.2-6.el7 for package: rhn-setup-2.0.2-6.el7.noarch
--> Processing Dependency: rhnsd for package: rhn-setup-2.0.2-6.el7.noarch
--> Running transaction check
---> Package rhn-client-tools.noarch 0:2.0.2-6.el7 will be installed
--> Processing Dependency: rhnlib >= 2.5.57 for package: rhn-client-tools-2.0.2-6.el7.noarch
--> Processing Dependency: python-hwdata for package: rhn-client-tools-2.0.2-6.el7.noarch
--> Processing Dependency: python-gudev for package: rhn-client-tools-2.0.2-6.el7.noarch
--> Processing Dependency: python-dmidecode for package: rhn-client-tools-2.0.2-6.el7.noarch
---> Package rhnsd.x86_64 0:5.0.13-5.el7 will be installed
--> Processing Dependency: rhn-check >= 0.0.8 for package: rhnsd-5.0.13-5.el7.x86_64
--> Running transaction check
---> Package python-dmidecode.x86_64 0:3.10.13-11.el7 will be installed
---> Package python-gudev.x86_64 0:147.2-7.el7 will be installed
---> Package python-hwdata.noarch 0:1.7.3-4.el7 will be installed
---> Package rhn-check.noarch 0:2.0.2-6.el7 will be installed
--> Processing Dependency: yum-rhn-plugin >= 1.6.4-1 for package: rhn-check-2.0.2-6.el7.noarch
---> Package rhnlib.noarch 0:2.5.65-2.el7 will be installed
--> Running transaction check
---> Package yum-rhn-plugin.noarch 0:2.0.1-5.el7 will be installed
--> Processing Dependency: m2crypto >= 0.16-6 for package: yum-rhn-plugin-2.0.1-5.el7.noarch
--> Running transaction check
---> Package m2crypto.x86_64 0:0.21.1-17.el7 will be installed
--> Finished Dependency Resolution
Package Arch Version Repository Size
rhn-setup noarch 2.0.2-6.el7 th_lab_server 87 k
Installing for dependencies:
m2crypto x86_64 0.21.1-17.el7 th_lab_server 429 k
python-dmidecode x86_64 3.10.13-11.el7 th_lab_server 82 k
python-gudev x86_64 147.2-7.el7 th_lab_server 18 k
python-hwdata noarch 1.7.3-4.el7 th_lab_server 32 k
rhn-check noarch 2.0.2-6.el7 th_lab_server 52 k
rhn-client-tools noarch 2.0.2-6.el7 th_lab_server 379 k
rhnlib noarch 2.5.65-2.el7 th_lab_server 65 k
rhnsd x86_64 5.0.13-5.el7 th_lab_server 48 k
yum-rhn-plugin noarch 2.0.1-5.el7 th_lab_server 80 k
Install 1 Package (+9 Dependent packages)
Total size: 1.2 M
Total download size: 1.2 M
Installed size: 4.8 M
Is this ok [y/d/N]: y
(1/9): python-gudev-147.2-7.el7.x86_64.rpm | 18 kB 00:00
(2/9): m2crypto-0.21.1-17.el7.x86_64.rpm | 429 kB 00:00
(3/9): python-hwdata-1.7.3-4.el7.noarch.rpm | 32 kB 00:00
(4/9): rhn-check-2.0.2-6.el7.noarch.rpm | 52 kB 00:00
(5/9): rhn-client-tools-2.0.2-6.el7.noarch.rpm | 379 kB 00:00
(6/9): rhn-setup-2.0.2-6.el7.noarch.rpm | 87 kB 00:00
(7/9): rhnlib-2.5.65-2.el7.noarch.rpm | 65 kB 00:00
(8/9): rhnsd-5.0.13-5.el7.x86_64.rpm | 48 kB 00:00
(9/9): yum-rhn-plugin-2.0.1-5.el7.noarch.rpm | 80 kB 00:00
Total 1.3 MB/s | 1.2 MB 00:00
Running transaction check
Running transaction test
Transaction test succeeded
Installing : python-gudev-147.2-7.el7.x86_64 1/10
Installing : rhnlib-2.5.65-2.el7.noarch 2/10
Installing : python-hwdata-1.7.3-4.el7.noarch 3/10
Installing : python-dmidecode-3.10.13-11.el7.x86_64 4/10
Installing : rhn-client-tools-2.0.2-6.el7.noarch 5/10
Installing : m2crypto-0.21.1-17.el7.x86_64 6/10
Installing : rhnsd-5.0.13-5.el7.x86_64 7/10
Installing : rhn-setup-2.0.2-6.el7.noarch 8/10
Installing : yum-rhn-plugin-2.0.1-5.el7.noarch 9/10
Installing : rhn-check-2.0.2-6.el7.noarch 10/10
Verifying : rhn-setup-2.0.2-6.el7.noarch 1/10
Verifying : m2crypto-0.21.1-17.el7.x86_64 2/10
Verifying : rhn-check-2.0.2-6.el7.noarch 3/10
Verifying : python-dmidecode-3.10.13-11.el7.x86_64 4/10
Verifying : rhnsd-5.0.13-5.el7.x86_64 5/10
Verifying : rhn-client-tools-2.0.2-6.el7.noarch 6/10
Verifying : python-hwdata-1.7.3-4.el7.noarch 7/10
Verifying : yum-rhn-plugin-2.0.1-5.el7.noarch 8/10
Verifying : rhnlib-2.5.65-2.el7.noarch 9/10
Verifying : python-gudev-147.2-7.el7.x86_64 10/10
m2crypto.x86_64 0:0.21.1-17.el7 python-dmidecode.x86_64 0:3.10.13-11.el7
python-gudev.x86_64 0:147.2-7.el7 python-hwdata.noarch 0:1.7.3-4.el7
rhn-check.noarch 0:2.0.2-6.el7 rhn-client-tools.noarch 0:2.0.2-6.el7
rhnlib.noarch 0:2.5.65-2.el7 rhnsd.x86_64 0:5.0.13-5.el7
Next step is to install your Spacewalk server’s ssl certificate on the client. This is a security measure which enables the client to verify that the server it is talking to is really the server it SHOULD be talking too.
The final step in the process is to actually register the client against the Spacewalk/Satellite server.
[toby@devops ~]$ sudo rhnreg_ks --serverUrl=https://manager.lab.tobyheywood.com/XMLRPC --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=1-lab.tobyheywood.com
This system is not subscribed to any channels.
RHN channel support will be disabled.
At this point, we float over to the Spacewalk server UI, and we should now see our client in the list of Systems;
Now, for those of you who have a keen eye for detail, you will have noticed in the screenshot above and the snippet of command line output at the time of registration, that the system isn’t currently subscribed to any channels. This is very easily remedied;
Click on the client in the system list
On the initial Overview screen, you will see a box – Subscribed Channels
Click Alter Channel Subscriptions
Now Select from the list under Base Software Channel – in my case “centos_7_base”
You should now see the channel listed under the heading “Software Channel Subscriptions”.
In addition you may have child channels created beneath your base channel.
And there we have it. Time to have a play and see what you can do by having a click around the tabs related to the system and the wider Spacewalk UI.
Following on from an earlier post, it would seem that the “Warning /dev/root does not exist” issue is not confined to “none” kickstart pxe booted installations as I had first thought.
I was working on a RHEL 7 installation using Red Hat Satellite 5.7 (upgrade to 6.x in the pipeline but bigger fish to fry right now), where we were re-using a lot of the RHEL 6 pxelinux kernel parameter’s.
Now as you may or may not know (if you have read my other posts on the topic), there are numerous Anaconda and dracut parameter’s that can be passed to the kernel in the pxelinux.cfg/default (or the mac specific) config file. The problem we had found was the existence off a ksdevice= parameter which pointed to eth0. In RHEL/CentOS 7, the ethernet device naming standard changes from ethX to ensX, which works out as follows;
en = Ethernet
sX = Slot X (where X is the physical or virtual slot number where the nic resides)
By default the first interface is used by anaconda/dracut/pxelinux, IF no option is specified. If however you specifically tell it to use something which fundamentally doesn’t exist, it WILL still try to use that… and fail! Miserably! And give you an error which ultimately seems kind of unrelated.
You have been warned!
As with many things in life, this served as a reminder that things change and that you can’t always take the “old” and reuse with the “new” without issue.
It would appear that I have been caught out twice now due to the way that nmtui (Network Manager Text User Interface) works. I have been messing around with various internal sandboxed networks in my VM environment and (I can only assume ion my haste), I have entered the IP address of a second NIC without full regard for the on screen prompts.
In nmtui, there is one field missing which is quite common in many other tools. Take a look at the following and tell me what’s missing;
So what field do you think is missing?
Now although the information is all on screen in the screenshot above, there is one thing that may not be obvious. In the Addresses field you specify not only the IP address but also the subnet mask in CIDR (which stands for Classless Inter-Domain Routing) notation.
IF, you happen to enter and IP address without thinking about it and don’t specify the netmask or CIDR, nmtui assumes that you are only referring to a /32, a.k.a. a netmask of 255.255.255.255, which for the uninitiated means just that IP. If assumes that there is nothing beyond that IP address. It’s world is only itself.
In the good old days where I used to configure the IP address via ifcfg-eth* files, I also remembered to enter the NETMASK= line, and therefore never had this issue.
Anyway rant over. Hopefully twice is enough, because if nothing else, if I have name resolutions errors in my logs again, I will be making sure my netmask is set correctly, before thinking that tomcat is having issues.
Featured image credit: Thanks to versageek for making the Network Spagetti image available on Flickr.com.
After having installed Spacewalk, got it working to a certain point and then found that there may have been issues with the installation, I thought it would be easier to simply re-install spacewalk onto a new virtual machine.
So following on from my how to article, I wanted to make sure that post installation, I had performed sufficient checks to confirm that there were no issues with the scheduler service or cobbler, as these were two things I had great difficulty trying to get working.
I guess it is also worth mentioning that the VM I am running spacewalk on has a single vCPU and 4GB of memory. For storage I have given it 40G which will do me fine. And as for the OS it is running CentOS 7 (1511).
So what should we check
Good question. The following is a rough list of all the services I confirmed as enabled, running and also that there were no horrible errors in the log files
taskomatic (a.k.a. the scheduler)
[toby@manager ~]$ sudo systemctl status cobblerd
● cobblerd.service - LSB: daemon for libvirt virtualization API
Loaded: loaded (/etc/rc.d/init.d/cobblerd)
Active: active (running) since Fri 2016-04-15 23:08:37 BST; 24min ago
└─13257 /usr/bin/python -s /bin/cobblerd --daemonize
Apr 15 23:08:35 manager systemd: Starting LSB: daemon for libvirt virtualization API...
Apr 15 23:08:37 manager cobblerd: Starting cobbler daemon: [ OK ]
Apr 15 23:08:37 manager systemd: Started LSB: daemon for libvirt virtualization API.
Apr 15 23:31:35 manager systemd: [/run/systemd/generator.late/cobblerd.service:8] Failed to add dependency on network,.service, ignoring: Invalid argument
Apr 15 23:31:35 manager systemd: [/run/systemd/generator.late/cobblerd.service:8] Failed to add dependency on xinetd,.service, ignoring: Invalid argument
The last two lines can be ignore. I believe this is purely due some references to the sysvinit scripts which are no longer used, and as you will see later things appear to be running fine (this time around)
Now, this one doesn’t appear to have been moved over to the new systemd environment and therefore we resort back to the good old sysvinit scripts and the service command to confirm this one is working;
[toby@manager ~]$ sudo service taskomatic status
RHN Taskomatic is running (13296).
Looking good so far
But can it withstand a reboot? Now that is the question. So I repeated the above steps again, just to confirm. I won’t bore you with all the details;
cobblerd.service – active (running)
httpd.service – active (running)
tftp.service – inactive (dead)
postgresql.service – active (running)
tomcat.service –active (running)
taskomatic – RHN Taskomatic is not running.
[toby@manager ~]$ sudo service taskomatic status
RHN Taskomatic is not running.
[toby@manager ~]$ sudo chkconfig taskomatic on
[toby@manager ~]$ sudo service taskomatic start
Starting RHN Taskomatic...
[toby@manager ~]$ sudo service taskomatic status
RHN Taskomatic is running (10870).
And for good measure I gave the machine another reboot, just to confirm the taskomatic service did start.
[toby@manager ~]$ sudo service taskomatic status
RHN Taskomatic is running (1278).
Oh Yeah! Now I’m a happy camper. And it’s time to re-visit the initial configuration part, which I shall post about shortly.
Image credit; Thanks to Mark Walsh for making the featured image called “Russell Street Court Cells – Padded Cell” available on Flickr.com.
Email. One of those things which is critical to business and depending on what you read and who you speak to, you may be led to believe that it is dying out in favour of instant messaging technologies. Well, as of right now, I can’t see it dying out any time soon, though it does come with a really annoying, frustrating, excruciating problem. Junk mail! Otherwise know as spam! It has once again become a bit of a problem for me, especially when the likes of yahoo begin to imped the flow of emails leaving one of the servers I have worked on recently unable to send emails to recipients with Yahoo accounts.
Now, don’t get my wrong. I don’t hold a grudge with any of the large email service providers for blocking spam which has sadly managed to be sent via a mail server I have some involvement with, but I do wish the world wasn’t such a bad place that we have to continuously fight on our email frontiers to protect our virtual/real name and brand.
So anyway. On with the show
Patched to the nines
Blocking of known spammers was enabled
Spamassassin was used as a secondary measure should the hard blocks fail
SPF is configured for all hosted domains
DKIM was not configured
DMAC was not configured
It *was not* an open relay
The time was not in sync, which didn’t help later down the line
The server’s “reputation” had been tarnished as someone had managed to start sending spam via the server through a vulnerability which was found in one of the websites hosted on the server.
After reviewing the advice from Yahoo, I started my investigation on how to approach this topic.
A bit of googling, led me to this web site; https://www.strangeworld.com/blog/archives/128. Now for me, there were a few things missing in the instructions, though complete were it was most needed. So recreating some of the good work done on the strangeworld.com website and providing my own twist on things here;
DKIM & ZDKIMFILTER Pre-requisites
In order to install zdkimfilter you need to manually compile the code and install (a .spec file will be included in future releases which should mean you can just use rpmbuild to create the RPM package for you). The machine I needed to install this onto did not have any compilers install for reasons of security, and so my list of pre-requisites was rather long and included more than I would have liked but needs must, and the compilers were removed afterwards. But anyway, here is the list of commands issued to get everything where it needed to be;
Note. The addition of the elrepo.org rpm is required as there are a couple of dependencies, which don’t appear to be available in the main CentOS yum repository.
Download, compile and install
Now, the original instructions I followed, (see reference material at bottom of this page), took me through the steps of manually compiling the code for and installing opendkim. After I had installed it, I found that opendkim is available in the CentOS base yum repo and so I could have avoided that step. You will be pleased to hear that I have not included those steps here (if you do need to know though, see this blog post – strangeworld.com).
Next up we need to obtain the tar ball containing the source code for zdkimfilter, which was downloaded from; http://www.tana.it/sw/zdkimfilter/. At the time of writing the current version was v1.5.
Now we have that lets work through the process of compiling and installing it.
tar zxvf zdkimfilter-1.5.tar.gz
sudo make install
Last on the list of installation steps is configuring zdkimfilter and making sure we have the correct directory structure, etc;
At this point, we should have everything installed and configured that we need, and can now proceed with the steps for generating the necessary private keys which will be used to sign our emails as they leave your courier email server and also the public key that will be published in DNS.
A word of warning. Not sure how important this is, but when looking at generating the keys using opendkim-genkey, I would advise that you choose your selector name carefully. Initially I set this to be the same as the domain. I then went to a couple of dkim testing websites and found that one of them didn’t like the “.” (dot) in the selector name. At this point I changed my plan slightly (in case this was a more wide spread issue) and made sure there were no periods in the selector name.
opendkim-genkey -b 2048 -d tobyheywood.com -D /etc/courier/filters/keys -s tobyheywood -r --nosubdomains -v
ln -s tobyheywood.private tobyheywood.com
chown root.daemon tobyheywood.*
(Just making sure ownership is correct)
chmod u=rw,g=r,o-rwx tobyheywood.*
(this will change the permissions on the tobyheywood.private file which contains the private key, and the tobyheywood.txt file which contains the public key)
It’s also worth making sure the directory /etc/courier/filters/keys has the right permissions;
chmod u=rwx,g=rx,o-rwx /etc/courier/filters/keys
And also if you are running SELinux, then it is worth double checking the SELinux security context associated with these newly created files. In my case, I didn’t have to do anything, but best to check.
Now you need to update your DNS zone file(s) with everything in the .txt file up to and including the last “)”. You could include the rest if you wish, but it’s purely a comment. Reload you zone and test.
Testing, testing, 1, 2, 3
I performed two types of test. The first was to make sure the record was returned correctly when queried by third parties and that there were no issues with the DNS side of things. The website I preferred was; https://www.mail-tester.com/spf-dkim-check as it was a nice and simple site and worked better than the unnamed first site I tried.
The second part of my test, was to send an email to a yahoo or gmail account to confirm that it was accepted and so that I could review the headers. This turned out to be a very good move! The headers of my first test emails showed there was no DKIM key present, which I thought was odd, but that was due to the umask and the keys directory didn’t have the execute bit set for group members. I have corrected this in the above [rough] instructions.
Another attempt at sending an email showed there was a valid key but that the time on this server was ahead by some 6 minutes. I then found that NTP hadn’t been enabled and so had to enable this in order to get the time aligned with other servers on the Internet.
My last test email, was a complete success. The headers showed that everything was as it should be. The DKIM signature had been accepted.
The next thing I need to read up on and implement for the domain in question is dmarc. But more on that later.