Red Hat Satellite Server – Fatal error in Python code occurred [[6]]

Posted on Leave a commentPosted in Deployment, Linux, Python, Red Hat Satelitte Server, RHEL 7, Spacewalk, System Administration, Systems Management

I have embraced Red Hat Satellite server in a big way over the past year and try to use it wherever possible though not for everything.

One of the features I started using to simply life whilst I look at other configuration management systems, was Configuration Channels.  These allow you to provide a central repository of files and binaries which can be deployed to a server during the initial kickstart server deployment process.

Some changes had been made a month or so ago, to ensure that a specific configuration channel would be included in future deployments by way of updating the Activation Key for that deployment type in Satellite server.  Seems innocent enough at this point.  It is worth noting that there were other configuration channels associated with this activation key.

At the same time I had also added a couple of packages to the software package list which were also required at time of deployment.

Now, I rely on scripts which have been deployed to a server to complete some post server build tasks.  The first thing I noticed after a test deployment, was a complete lack of any scripts where I expected them to be.  The configuration channels had created the required folder structure but had stopped completely and had gone no further.  The error the Satellite server reported back to me was… well not massively helpful;

Fatal error in Python code occurred [[6]]

Nothing more, nothing less.

At this point I started trying to remember what I had added (thankfully not to hard as I document things quite heavily 🙂 ).  Here is roughly the steps I took to confirm whether the issue resided;

  • Remove the additional packages I had specified for this particular build – made no difference
  • Remove what I the most recently added configuration channel – made no difference
  • Tested another Red Hat Enterprise Linux 7 build (not using this particular kickstart profile) – success, so the issue would appear to be limited to this one profile
  • Remove the other configuration channels that were added some time before the last one was added – failed, still the configuration channels would not deploy. But wait, there was light at the end of the tunnel!

But, following this last step, the error message changed, from something not very helpful to something quite helpful indeed!  The message stated that permissions could not be applied as per those stipulated against specific files in the configuration channel.

So it transpires that it was a permissions resolution issue. Well, more a group resolution issue really.  There were a couple of files which were set to be deployed with a specific group.  The group in question is served from a LDAP server and the newly built machine wasn’t configured at that point to talk to the LDAP server, for this particular deployment we didn’t want auto registration with the LDAP services.

So the lesson here is make small changes, test frequently and make sure you document what you have done.  Or use a configuration management system which is version controlled, so you can easily roll back.

Just so we are clear, I was running Red Hat Satellite Server 5.7 (full patched) on RHEL 6.8 and trying to deploy RHEL 7.3.  My adventure to upgrade Satellite server to version 6.2 will be coming to a blog post soon.

So, it would appear this story comes with a lesson attached (free of charge) that all should take note of – “Always make one change at a time and test or as near to one as you can”.

Featured image credit: Charly W Karl posted e.Deorbit closing on target satellite on Flickr.  Thanks very much.

Warning /dev/root does not exist – The Devil is in the Detail

Posted on Leave a commentPosted in CentOS 7, Deployment, Linux, Red Hat Satelitte Server, RHEL 7, Systems Management

Following on from an earlier post, it would seem that the “Warning /dev/root does not exist” issue is not confined tononekickstart pxe booted installations as I had first thought.

I was working on a RHEL 7 installation using Red Hat Satellite 5.7 (upgrade to 6.x in the pipeline but bigger fish to fry right now), where we were re-using a lot of the RHEL 6 pxelinux kernel parameter’s.

Now as you may or may not know (if you have read my other posts on the topic), there are numerous Anaconda and dracut parameter’s that can be passed to the kernel in the pxelinux.cfg/default (or the mac specific) config file.  The problem we had found was the existence off a ksdevice= parameter which pointed to eth0.  In RHEL/CentOS 7, the ethernet device naming standard changes from ethX to ensX, which works out as follows;

  • en = Ethernet
  • sX = Slot X (where X is the physical or virtual slot number where the nic resides)

By default the first interface is used by anaconda/dracut/pxelinux, IF no option is specified.  If however you specifically tell it to use something which fundamentally doesn’t exist, it WILL still try to use that… and fail!  Miserably!  And give you an error which ultimately seems kind of unrelated.

You have been warned!

As with many things in life, this served as a reminder that things change and that you can’t always take the “old” and reuse with the “new” without issue.

Featured image credit:  Thanks to (Elba) Dave Shewmaker for taking the rather weird picture which resembles The Devil in the Detail and posting it on Flickr.