State of Play revisited

churchill150I mentioned some time ago that Churchill was looking pretty sick since no RHCSL 6.x distro with a version number greater than 6.6 seemed to deliver its software on a single DVD any more, and since multi-DVD installs break Churchill, Churchill was doomed.

But I was (happily) quite wrong on that point. It turns out that, in fact, CentOS is the only RHCSL distro that has chosen to package its distro in such a way that it breaks Churchill.

I’ve checked OEL 6.6, 6.7 and 6.8; and then I checked Scientific 6.6., 6.7 and 6.8; and I even checked genuine Red Hat 6.6, 6.7 and 6.8… and every single one of them works fine with Churchill.

So, as I mentioned last time, that now means Scientific Linux is the distro that Churchill will assume is in use if you use its ‘speed keys’ to build your Oracle server nodes. (Rather more bluntly, it means I’ve stopped using CentOS at all, for anything). You can still use OEL or Red Hat if you prefer, by spelling out your requirements on the boot strap line. But use a speed key to save some typing, and SL will be assumed.

And, of course, if you want to use CentOS 6.7 or 6.8 to build your Oracle servers using Churchill, forget it: because they just don’t work.

Why is the new default Scientific Linux and not, say, OEL? Simply because it’s (a) free to obtain without signing up for anything and (b) it’s not as butt-ugly as OEL when you boot it up (white text on a red background is neither pretty nor particularly legible).

Other than this issue of CentOS stopping working, the latest version of Churchill (we’re now up to version 1.6) introduces a rather more radical change of tack: no more support for Oracle 11g installations, of any sort, whatsoever. Of course, version 1.5 is still available which does still do 11g quite nicely -but 11g is so long in the tooth now, and completely out of support unless you happen to be using, for which an expensive paid support agreement is needed before it can even be downloaded, that it seemed a waste of time to keep 11g support in newer Churchill releases. So, 12c only it is.

Which of course means that Churchill’s hardware requirements have just gone through the roof to match Oracle’s own. A 2GB VM just doesn’t cut it any more. The new hardware minima are: 5GB RAM and a 40GB hard disk. So building a 2-node RAC and a 2-node Active Data Guard setup on your desktop now basically demands a 32GB desktop… and no laptops need apply.

All of which in turn means that I think it might be time to re-write the Churchill documentation (especially as it currently exists only as pages on the old version of this site): take the opportunity to spruce it up, refresh it, make it part of the native WordPress site and so on. Expect something soon-ish.

You can’t kick this around anymore…

nixonscan-finalRichard Nixon is the US president I admire the most, I think. He had a grasp of strategic thinking that few have matched. My admiration is often a source of amusement (and embarrassment) to my American friends, though, because that grasp of strategy failed him spectacularly during the Watergate affair, resulting in his resignation and disgrace.

But from my father, I sought, and was granted, special permission to watch his resignation speech, live, at 2AM London time in August 1974. I was a pretty committed 10 year-old! I also strongly suspect that without Nixon, none of us might be here today: the Cold War could have got very hot and very nasty on numerous occasions, until Nixon’s policy of détente made us all a lot safer.

Anyway, my fascination with the man lead me, several years ago now, to look to obtain his autograph (as one does with heroes). But they were (and are) not cheap, and ToH would not permit money-siphoning to take place on such a grand scale, despite my entreaties that a great man warrants grand money-wastage! Cheaper ones can be found if you look long enough, though-but I wasn’t that diligent back then, so everything seemed awfully pricey.

And then I happened to stumble across an auction on Ebay, for a letter allegedly signed by Nixon whilst he was serving as Vice President -so back in 1960. The asking price was about US$89, and no international shipping.

Well, at that price, it couldn’t be genuine, could it?! But the lady selling it said it had been sent to her father for some reason, he had died recently, she had no use for it and therefore wasn’t asking the world for it either. However, she couldn’t substantiate it more than that and knew nothing about its authenticity. But she would ship it to me in Australia if I paid some token sum or other (maybe an extra US$25).

And that is how I came to acquire my Nixon autograph for the piffling price of about US$115. It eventually arrived safely in the post -and this was my first chance to actually look and touch what I’d paid for. Well: I wasn’t disappointed. It certainly looked genuine (the signature is strong and definitely from a fountain pen). It also felt genuine: the paper is definitely old, the typewriting is definitely from a real typewriter.

Looks can be deceptive, of course! Nevertheless, the thing was framed and put in a place of honour on my study wall, where it has remained to this day.

But was it genuine??! It would bug me occasionally.

So early this week, I finally decided to do something about it: I sent the scanned copy of it you can see at the top of this piece to the Nixon Presidential Library, in Yorba Linda, California. I asked them if they had any records proving that, as the letter claimed, Nixon had been in Akron, Ohio on October 1st 1960 -and whether they could tell me anything about the addressee, Mr. Giampetro.

Their website is not the best! They don’t appear to know how to get https certificates to work, either! So I wasn’t hugely hopeful… but efficiency lurks in other corners of the library, it would seem, because this evening, not two days after I emailed, I received a very nice email from their archivist, one Dorissa Martinez, saying (and I quote):

According to our Pre-Presidential Materials (Laguna Niguel) Appearances (Series 207) finding aid, Vice President Nixon attended a rally at Memorial Hall in Akron, Ohio on October 1, 1960.

After searching through the Pre-Presidential Materials (Laguna Niguel) General Correspondence (Series 320) collection and Campaign 1960: PPS 57: Election. 1960. Post Election Correspondence, Acknowledgments, and Thank You’s, box 5, folder Thank you Letters Completed – Nationwide, I was unable to locate materials relating to Frank Giampetro.

I’m not sure I made entire sense of either sentence! But the general gist of it is that if you poke around the Pre-Presidential Materials long enough, you can indeed find evidence that Nixon was in Akron, Ohio on the date the letter claims he was. It doesn’t prove my letter is genuine… but it goes a long way to reassuring me on the point anyway.

It’s a shame there weren’t records that Frank Giampetro was, for example, a Republican Party hired driver, routinely used to transport any bigwigs that came by Ohio… so that part of the mystery remains. I’d certainly like to find out more about him (but the letter stays with me, come what may!)

Anyhow: I don’t think it would have mattered to me either way, but it’s nice to have my own little bit of near-confirmed Nixon-alia in front of me as I type. Let’s just hope greatness rubs off and shady dealings don’t!

State of Play

churchill150It has been almost a year since I did anything to my Churchill automation framework. (If you didn’t know or realise, Churchill lets you easily create virtual multi-node RAC and Active Data Guard environments).

I took a look at it lately to see about refreshing it. I discovered that whilst Churchill’s speed keys will assume that you’re using CentOS 6.6, CentOS has actually released versions 6.7 and 6.8 since the last Churchill refresh (as have Red Hat, Scientific Linux and Oracle, of course)! It is also still the case that no version of Churchill works with a RHCSL 7.x O/S (because systemd screws up network automation).

No worries, I thought to myself: tweak a few things and at least bring Churchill up-to-date-ish with version 6.8 as the new speedkey default.

No chance!

In their wisdom, the CentOS developers decided to split the main DVD release of 6.8 over two DVDs. That means Churchill’s trick of copying its own installation media onto its own hard disks (for subsequent automated network installs onto the RAC nodes to use) fails, because it can only copy one of the two install DVDs. (There is a special, 6.8-only Dual Layer DVD you can use, and that would work… but I decided not to go there for now).

This, combined with the systemd debacle, basically means that Churchill is dead in the water, and I therefore have made only one more tweak to it -whereby Scientific Linux version 6.7 is assumed to be the speedkey default- and after that, I won’t be maintaining it further.

Something new is required to automate 12c RACs on RHCSL 7.x environments… and I don’t know what that will be yet! Watch this space, I guess…

Fun Fedora 24

Just as my playing with the new Linux Mint release begins, so the Fedora team finalise a new version of their distro: Fedora 24 was released on 21st June.

It’s still very blue; it’s still very Gnome-y and therefore pretty awful as far as I’m concerned and I wouldn’t personally touch it with a feathered hat-band, let alone a bargepole.

But it’s out and therefore my Bogart preinstaller script, which makes Fedora a suitable platform for running Oracle Enterprise Edition, needs a run in the park to make sure it still works with the new version. Happily it does without any substantial changes at all.

However, I took the opportunity to do two things with Bogart. One was to remove its ability for preparing for an 11g installation. I know is still supported, but you can’t get hold of that without a support contract; and if you’ve got a support contract, you won’t likely be wanting to run Oracle on an unsupported platform like Fedora! Meanwhile, any other version of 11g you can get your hands on has long-since been de-suppported… so Bogart is now 12c only.

And that means, two: I’ve re-written the Oracle-on-Fedora article to reflect it’s new only-12c-ness.

The revised article is here, and the updated Bogart preinstaller script is here.

Minty Fresh

Mint2Having just finished a push for domestic consistency by installing Ubuntu 16.04 on practically every machine I control (which therefore definitely excludes a certain Significant Other’s Windows PC!), I now see that the Linux Mint crew have just released a new version of their green and fragrant distro.

Never one to let the opportunity of a new distro installation pass me by if I can help it, I have accordingly just installed the Mate version onto my ancient(circa 2009) laptop… and it’s running nicely. Pretty slow, of course, but hardly unbearable. I probably don’t want to be doing Blender renders on it any time soon, but it’s fine for a bit of web editing, browsing, video playing and photo up-touching.

It also has its windows controls (maximise, minimise and close) on the right side of the window title bar (which is to say, yes it’s on the right-hand side, but it’s also the correct side as far as I’m concerned!) It is a small cosmetic change from vanilla Ubuntu, I guess; but it’s enough that it might persuade me to deploy it on my main PC instead of the current Ubuntu install. Time will tell.

I noticed in passing that though I had written simple installation scripts for Oracle 12c for a lot of other distros I’ve dabbled with over the years, I had curiously neglected to write one for Linux Mint, of whatever vintage.

So I’ve rectified that. On the Oracle articles page is a new one for installing on Linux Mint 18. I’ve run out of inspiration for my script names now, so this one is just called Mentha -which, as we all know, is the Latin for ‘mint’.

I tested it on both the Cinnamon and Mate spins, but the screenshots in the article are all from Mate (for absolutely no reason at all, other than I happened to have it handy at the time the screenshots needed to be taken).

I should perhaps mention that I’m trialling the use of new ‘slideshow’ technology in this new article: it means the bazillion screenshots don’t take up nearly so much room on the screen and fade neatly from one to the other as you step through the slideshow. The image captions provide the instructions. It looks good -but there are at least two issues with it. One, if you’re using script blockers, the slideshow won’t work properly. And two, the plain-text version of the article (obtainable by clicking the ‘print’ link at the very end of the it) displays all the screenshots sans instructions, which is somewhat less than useful! If that is an issue for you, let me know and I’ll re-think… I am not entirely sure how many people bother with the plain-text versions of articles, so I don’t know how much of a deal-breaker this is. As I say, tell me (in the comments) if it is one for you…

Server Error: 550

RH_bestpractice_img_commonBouncesAs I mentioned last time, I have abandoned efforts (for the moment) to get my own postfix/dovecot/etc/etc mail server working due to security issues that are non-trivial to work around. I also mentioned that I only found out about the security issues after I’d written six articles about putting the pieces together!

I suppose it was inevitable that a number of people would ask me to publish the six articles despite the end result not being entirely satisfactory from a security point of view… so I have, and they are available from the Linux Articles page.

Return to Sender

Postfix_logoI have spent the past week or so -and not an inconsiderable amount of coin- trying to build a CentOS7 server that uses Postfix, Dovecot, Spamassassin, Amavis and Roundcube so that I can finally ditch the ever-more-privacy-intrusive mail services from the likes of Gmail and Outlook. It is a madness that comes over me once every year or so. The enormous, steaming pile of disparate software pieces that have to bolted together just-so to get anything working at all has never really put me off trying… but my previous efforts, though functional, haven’t really been fully functional and never took security too seriously.

This time I got really close to doing it properly. Everything worked fine after just the seventh installation-from-scratch attempt, and six nice how-to articles ready to publish. 95% of the work had been sorted out on the first pass, I have to say -but the missing 5% was the very devil to get right. It was all to do with delivering Spam (because no spam detection algorithm is perfect), but into a junk mail folder rather than the main inbox. The tool to do that is called ‘dovecot-pigeonhole’ and the protocol involved is called ‘sieve’… and configuring sieve and dovecot in a way that actually works is a nightmare. But I did it. It worked. Spam moved all by itself to a place where I could save it if I wanted to or delete it if I didn’t.

And then I logged on with the wrong password… and Roundcube (the webmail interface) let me in regardless!


Turns out that by design and default, MySQL (and thus Dovecot and thus Roundcube) uses ‘CRYPT’ as its password encryption technology, which is based on DES, and (here’s the punchline) only uses the first 8 characters of your supplied password. So yes, I had mis-typed the 23rd character of my 26-character long password (it happens!)… but Roundcube had let me in anyway because the first 8 characters had been typed in an impeccably accurate manner.

Several things to say about this. The first one is that it’s bloody annoying! Second, it’s bloody stupid. And third, after an hour reading about how you can configure things to do SHA512-CRYPT (which doesn’t chop your passwords in thirds without telling you), I spent 45 minutes trying to implement it without success before giving up in utter frustration.

I am now using again!

Of the 98 hurdles between me and a working, secure, web-accessible, flexible, virus-blocking, spam-delivering email server, I had managed to leap, more or less gracefully, 97 of them. But truncating passwords is too much like plain stupidity to make the effort involved in limping over the 98th just too much bother.

I wonder if there is a non-postfix, non-dovecot means of doing a mail server… I have a spare server in Paris to make use of, after all…

All done

finish01The move to a new server has now completed, and if you’re reading this then I assume everything worked as intended.

There are a few minor, known issues: anything I’ve ever linked to in my file repository at is now broken, because it was simpler to create a new Owncloud file respository from scratch than to try to migrate the thing over. Fortunately, there are only a dozen or so such links: they should be sorted by the end of the day.

Otherwise, it’s all lovely, clean and fresh and working well!

(But if you do encounter glitches or problems with anything not working, feel free to let me know in the comments).

There was one nasty surprise that arose as a result of switching from ye olde, stable CentOS (MySQL version 5.1.something) to flash-harry Ubuntu (MySQL version 5.7.something). Between those two versions, MySQL invented something called “Strict Mode” -and in 5.7, this actually became the default operating mode. Specifically, the “STRICT TRANS TABLES” mode was enabled -and this governs the way MySQL reacts to things like implicit data conversions in a SQL statement. In non-strict mode, for example, an attempt to set a VARCHAR column to a value of 1 would raise a warning, but not fail. In strict mode, it fails. (As any fule knowe, if you want a varchar to contain the string value 1, you need to wrap your ‘1’ in quotation marks!)

The relevance to my server move? As per this article, I now re-compute the comment counts for my posts every time I publish a new one -and my query to do that is this:

UPDATE dizwp.wp_posts wpp
(SELECT comment_post_id AS c_post_id, count(*) AS cnt FROM dizwp.wp_comments
 WHERE comment_approved = 1 
 GROUP BY comment_post_id) wpc
SET wpp.comment_count=wpc.cnt
WHERE wpp.post_type IN ('post', 'page')
      AND (wpp.comment_count!=wpc.cnt OR (wpp.comment_count != 0 AND wpc.cnt IS NULL));

Spot the error? Neither did I for a long time!! However:

mysql> desc wp_comments;
| Field                | Type                |
| comment_ID           | bigint(20) unsigned |
| comment_post_ID      | bigint(20) unsigned |
| comment_author       | tinytext            |
| comment_author_email | varchar(100)        |
| comment_author_url   | varchar(200)        |
| comment_author_IP    | varchar(100)        |
| comment_date         | datetime            |
| comment_date_gmt     | datetime            |
| comment_content      | text                |
| comment_karma        | int(11)             |
| comment_approved     | varchar(20)         |

Spot it now??

COMMENT_APPROVED is declared to be of type varchar(20).

My update statement says: WHERE comment_approved = 1

The WHERE clause is treating the number ‘1’ as a number, because it hasn’t any quotation marks around it. That is not good in strict mode when the column you are querying is a VARCHAR. 🙁

Two solutions, then: disable strict mode by editing /etc/mysql/mysqld.conf.d/mysqld.cnf and appending the line:


…after which you do a service mysql restart and the whole of MySQL is now running in non-strict mode and will do implicit data conversions without falling over.

Or you just re-write the original update statement to put quotation marks around the 1 in the WHERE clause:

WHERE comment_approved = '1'

I assume that WordPress knows all about strict mode and doesn’t need it disabled, therefore the addition of the quotes to my query seemed the more sensible option… but I imagine this is one of those niggly little issues that is quite likely to break an awful lot of MySQL applications if you’re not careful!

Anyway: apart from that, no major damage done, and I’ve edited the original article to add mention of this not-so-minor potential deal-breaker!

Un nouvel ordinateur

onlinelogoFor over a year now, this site and its sister have been hosted on a dedicated server hosted in a Parisian data centre, leased from, whose technical support has been excellent the few times I’ve needed to contact them, and whose uptimes are similarly unimpeachable. They get a big thumbs-up from me and I’d recommend them to anyone who wanted well-specified and extremely economical dedicated servers.

For the princely sum of €16 a month (or about AU$25), I get 1TB of disk space, 8GB RAM, a 4-core hyperthreaded CPU, 1Gb/sec networking and unlimited bandwidth. It was (and remains) an exceptionally good deal, I think (as compared to, for example, Linode with whom I previously hosted). The French also have the advantage of not kow-towing to the demands of the NSA (so far as we know!), which makes my paranoid self a little happier than I was with Linode 🙂

It pays to keep an eye on their product offerings, however, because I just noticed that they are now offering 16GB RAM for exactly the same price as I was paying for my 8GB, plus they’ve more than doubled the networking to a 2.5Gb/sec connection. (The specific product offering is called a “Dedibox XC 2016“; I decided to stick with 1TB of spinning SATA disk instead of opting for their 120GB SSD near-equivalent offering).

I asked about the possibility of upgrading my current box to their new specifications, but it’s not possible (and given we’re talking about physical servers in a rack, I’m not surprised). If you want to upgrade, you have to buy a new server of the new specification, move everything across to it and then cancel the original, now poorly-specc’d server.

Which is what I’m currently in the middle of doing.

I’ve made life a little harder for myself by building the new server as an Ubuntu 16.04 server rather than the CentOS 6.7 that adorned the original. I was rather well-practised at hardening CentOS installations; now I have to learn how to do it for an entirely new distro. Let’s hope I get it right, eh?!

If I can get the new server built, hardened and all content transferred across by June 20th, I can cancel the old server without having to pay an extra month. If not, I have to continue paying for it until the end of July. Fingers crossed, then, and I may only have 20 days of overlap to pay for.

Naturally, there will be some DNS updates to point things at the new servers if I get it right in time, and accordingly access to this site might get a bit tricky from time to time. Bear with me please: I’ll post a final confirmation when everything is across OK, but no doubt I’ll manage to screw things up a bit before I finally get it all right!


wpress01I think I’ve used just about every blogging platform out there in my time. The remembered terrors of Drupal and Joomla still keep me awake at nights.

I keep coming back to WordPress, though. It’s pretty easy to install, administer and use… and the results, visually, are appealing.

Unfortunately, everyone else keeps coming back to WordPress, too. Including hackers, crackers and other assorted ne’er-do-wells. As probably the world’s most popular blogging platform, WordPress starts off as a big target. Given that it’s written in PHP, however, it is also a very vulnerable one: one bad extension can open you up to all manner of nasties. It’s happened a lot of late.

Which gives me nightmares: I like to think that being self-hosted, I’m able to administer a tighter ship than was true for some of those cases, but I’m probably kidding myself. Just as living where I live, one has to accept that one day, I’m likely to lose the house in a bush fire, so I suspect that I ought to resign myself to the fact that, running a WordPress blog, one day I’m going to get hacked.

In the case of the house, acceptance of the inevitable means making sure you’re fully insured; that all your passports and birth certificates and similar papers are immediately to hand to grab as you flee; that anything that has huge sentimental value is similarly ready to accompany you on a quick departure… and that everything else is just ‘stuff’ and material possessions can be re-purchased.

In the case of the blog, accepting the inevitable has (for me) meant making plans to host the blog at home as the primary site; it only gets to the public webhost when I’m ready to copy it there. Being a copy that’s not accessible to the Internet means when the hoodlums strike, I can hopefully re-install a fresh, clean operating system on my public web server and copy everything up to it afresh. It will be inconvenient, sure; but at least my work will be safe from total loss and the nightmares can subside a bit.

As an added bonus, my websites not only reside on my new RAIDZ zpool, but are copied separately to my two HP servers, which also run RAIDZ pools… so that’s at least three copies of my work handy in the house. And there are two offsite backups of those HP servers which get refreshed around every 3 months. So, if the worst came to the worst: should hackers wipe my web server, my home PC explode at the same time, and two HP servers both suffer catastrophic 2-disk failures that same day… well, I might want to end it all for other reasons (it sounds like it’s a bad hair day plus infinity!), but I’d stand to lose only a couple of months of blogging. Which I could probably deal with.

So, I wrote an article or three to explain how I did it; I thought it might be of interest to other WordPress users out there!

Sleep tight…