I’ve been poking around with CentOS 7, Red Hat 7 and Oracle Enterprise Linux 7 extensively in the past couple of weeks, in the hope of producing a 7-compliant version of Asquith. For the sake of the rest of this post, lets agree to call all those distros, generically, Enterprise Linux 7.x
It’s been quite a ride, because an awful lot has been changed in the transition from Enterprise Linux 6.x to 7.x. I’ll list just some of the differences that have specifically tripped me up here:
How you invoke a Kickstart installation in the first place has changed
How you do firewalling has changed (firewalld not iptables)
The packages and package groups has changed
The way you configure iSCSI targets has changed, dramatically, and uses an interactive shell that isn’t suitable for scripting and doesn’t work with Kickstart
The way you disable and enable, stop and start services has changed (systemd v. init scripts)
The way network devices are named is now “intelligent”… and completely borks Kickstart
I’ll just explain a little more about that last point, by the way, since I’ve not mentioned it at all in any previous blogs. The gist of it is that you probably know and love your network interfaces as things like “eth0” and “eth1”, and have done for years. But they aren’t called that any more. Oh no. Instead, you get names such as “enp0s3” and “eno16777736” …and (this is the particularly cunning bit): you get different names depending on what your hardware and your BIOS is capable of.
The idea behind the change is logical and admirable in and of itself: in a server with two Ethernet cards, you were never entirely sure which one would pick up the “eth0” designation and which the “eth0”. Whereas now, the names are bus/slot dependent and are thus assigned deterministically: the card in slot 3 gets the ‘s3’ name and the one in slot 4 gets an ‘s4’ name. Simple, although you won’t know what your interface is going to be called until after the installation has itemised all your hardware and assigned the appropriate names.
The trouble is that in Kickstart, we used to do this sort of thing:
network --device eth0 --bootproto static --ip 192.168.8.101 --netmask 255.255.255.0 --nameserver 192.168.8.250 --hostname alpher.dizwell.home
network --device eth1 --bootproto static --ip 10.0.0.101 --netmask 255.255.255.0 --nameserver 192.168.8.250 hostname alpher-priv
That is to say: assign one set of network attributes to an interface called “eth0” and another to one called “eth1” …and do those assignments before the installation has completed.
So how do you write equivalent lines for an Enterprise Linux 7 Kickstart script when don’t know whether your interfaces are going to be called “enp0s3” or “eno16777736” or something completely different until after the installation has finished? You can’t. It’s just impossible to write one Kickstart script to run on any hardware now, because you won’t know what device names are ahead of time.
A case in point: my laptop and my home desktop PC, running a virtual machine in VMware Workstation, both produce Linux guests that end up with a network interface called “eno1677736”, but my work PC (also running VMware) produces guests that have a “enp0s3” network interface. One Kickstart script cannot do duty for both environments… and I have no idea what other variants my readers and would-be Asquith users might end up with, so I can’t even start taking them into account!
It’s a mess. In plain words: Enterprise Linux 7 breaks Kickstart installations.
I’ve had to work around it for now by reducing the Kickstart network line down to:
…which doesn’t attempt to configure networking interfaces at all. Later on in my Kickstart script, in the %post section, I then cheat like mad and run this:
for f in `ls /sys/class/net`; do
if [[ $f != "lo" ]]; then
cat > /etc/sysconfig/network-scripts/ifcfg-$f << EOF
…which simply writes a network configuration file for any non-loopback interface it finds listed in the /sys/class/net directory. This is taking place after the installation has all-but completed, so by that stage the finished interface names should be available. As that stands, it will write the same configuration for both interfaces in a 2-interface server, which is obviously not right… but a little bit of bash if-then-else’ing should see that right. For Asquith itself, which only has one network interface to worry about, this code as written will work no matter what interface names your installation decides to bestow upon your guest.
But it’s not “right” and it’s certainly not elegant, and the fact can’t be dodged that in their eagerness to embrace “meaningful network interface names”, the Enterprise Linux developers broke Kickstart. Hopefully, they will un-break it before too long.
Anyway… all these changes I’ve been whinging about of late apply equally well to all three variants of Enterprise Linux that I’ve been working with. It doesn’t matter whether you use CentOS, RHEL or OEL: Kickstart (for example) will struggle to configure your network interfaces correctly in all of them.
I do, however, have a special word of opprobrium to hurl uniquely in CentOS’s direction: it’s the only one of the three distros that decided it was more important to have a word processor on your Enterprise Linux server than a working Oracle database.
Let me explain: When you register for the Red Hat trial and download the 3.4 GB Red Hat 7 ISO; or when you download the Oracle Enterprise Linux 4GB ‘V46135-01’ ISO…. in both cases you end up with a single DVD image which contains a mix of 32-bit and 64-bit libraries/packages. As you know, Oracle’s Linux installs still require a mix of 32-bit and 64-bit packages to work properly (for example, you need glibc-x86_64 and glibc-i686 before things will compile correctly during the ‘linking phase’ of the Oracle database installation). So Red Hat and OEL both provide distro installation media which can satisfy those requirements.
But if you download the 4GB CentOS 7 installation DVD, you get pure, 64-bit only packages. No .i686 software exists at all, and thus no Oracle software installations are possible with it. I asked on the CentOS forums why they decided to package things up quite differently than their upstream vendor (i.e., Red Hat) did and the only explanation someone offered was that “to make room for the LibreOffice software, they had to ditch the i686 libraries”. I’m not sure if that’s so (a DVD ISO can be 4.7GB, so there’s room for 700MB of extras on the CentOS DVD even as it stands), but if it were true, it’s a weird choice: we package our Enterprise Linux distro so that you can run a word processor, but not an Enterprise-class database. You figure the logic of that, because I can’t see any in it.
You can, of course, install Oracle database software on CentOS by the “simple” expedients of either (a) connecting your server directly to the Internet and downloading the relevant 32-bit packages with yum; or (b) downloading the CentOS “Everything” ISO, instead of the plain-vanilla “DVD ISO”. The Everything ISO is 6.7GB in size and does include the i686 software packages you’ll need. But that means it’s nearly 3GB bigger than OEL or RHEL’s Oracle-ready equivalents.
I shall be interested to see how Scientific Linux do their packaging when the time comes (they are currently stuck at version 6.5, so I don’t know when or if SL7.0 will be making an appearance).
In the meantime, I shall have no choice but to strongly recommend NOT using CentOS 7 as a platform for Oracle databases. I’ll be switching all my development work to OEL 7.x, which is Oracle-database-ready AND can be downloaded and updated for free. CentOS just seems too weirdly and obtusely different from the other Enterprise Linux distros to be worth bothering with at the moment.
Updated to add: This isn’t the only point at which CentOS diverges in annoying ways from Red Hat or Oracle’s treatment of what is supposed to be essentially the same distro: try doing an lsb_release -r to see what version your distro reports itself to be. Red Hat reports 7.0. OEL reports 7.0. CentOS, however, decides it will be clever and report 7.0.1406. Version number reporting is important to Asquith, because it determines where you’ll fetch your software from when building client servers. Having one distro decide to be different from all the others is therefore, frankly, rather annoying!