Blog Archives

Blog entries related to server maintenance

How not to write an installer

OK, this is driving me crazy.  ISPConfig is a great piece of software.  It can manage your web sites, and related services (such as DNS, FTP, mail, etc.) well, and it handles the config files with pretty good care, and it rarely breaks.  But, the installation experience is so twisted that I can’t bear.  And if I would only have to tolerate this at fresh installs, then I’d just try to live with it, but facing the same old problems once more each time I try to update the installation is way beyond acceptable.

After you download their latest release, and untar it in, let’s say, /root, it creates a folder named install_ispconfig and then you’re just supposed to cd into that directory and issue a ./setup command.  Then the setup application asks you some questions about your system’s config1 (giving you a choice for an expert setup mode, in which it asks you all kinds of questions about all of the services installed on your system) and then it starts compiling Apache/PHP/OpenSSL/etc.  And if you’re doing an upgrade, you only have to choose your language and agree to their licensing terms, no further questions appear on the screen.  That sounds easy, right?

Wrong.  The installer is written with the assumption that everything will work right the first time.  If you answer incorrectly on any of those questions, or if something interrupts the installation process (be it as simple as an SSH session being disconnected, assuming you’re not using screen), then what are you supposed to do?  Just re-run ./setup and hope it picks up from where it left last time, right?

Wrong.  You’re supposed to delete the whole install_ispconfig directory2, and start from scratch!  Now, that doesn’t sound like a big deal?  Consider this.  I was updating ISPConfig on a server today, and nearly at the end of the install session, I got a number of weird shell error messages whining about unrecognized commands, and an Enter Password prompt.   Obviously I didn’t know what password to enter, so I tried the first password coming to my mind, the ISPConfig administrative password.  But it turned out that I was wrong.  Checking out the error messages, I noticed it should be the MySQL password.  Turns out that the ISPConfig installer tries to invoke the mysql utility with the password specified on the command line, and special characters in the password could confuse the shell, causing those error messages.  I found this after retrying the whole setup process (including half an hour to rebuild all the packages from the source) four times in a row.

You’d think that an installer should be able to handle such an error along the way?  You’d think that it should be able to use the results of those steps which were completed successfully in later runs?  You’d think that the compilation and configuration steps could be separated in the installer so that failing one does not abort the other?  If you’re working with the ISPConfig installer, you’d be wrong.  Come on, guys, you could do better.


1 It should be noted that strangely enough, it never asks you where you want ISPConfig to be installed.  It uses /root/ispconfig and /home/admispconfig, and you don’t get a say on that.  I still remember the first time I installed ISPConfig: I renamed the extracted directory to /root/ispconfig, and ran the setup from there, and wasted hours before I figured out that it’s the name of the directory which causes the installer to fail…

2 Actually the newer versions of their installer seems to delete that directory itself when something goes wrong, and doesn’t care to print a single message on the console.  Imagine how surprised you’d be when the installer quits, and you ls inside the install_ispconfig folder to see that the directory is empty!  And if you cd up and ls again, you’ll see there’s no install_ispconfig directory at all!  Hilarious!

Posted in Blog Tagged with: ,

Linux server monitoring

Server monitoring is something server admins put off until, well, it’s too late.  Server admins have all experienced clients calling in to let them know that a service on their server is not working.  This can be unfortunate and embarrassing at the very least.  I thought I’d document a number of utilities I use to monitor Linux servers.

  • Snort.  Snort is a NIDS and NIPS, and one of the most famous network security tools.  As a server admin, you should already know Snort.  I usually use BASE alongside Snort on my servers.
  • OSSEC.  OSSEC is a HIDS, which can perform a number of useful monitoring tasks, such as rootkit detection, automated log analysis, and file integrity checking.  In addition, it provides instant alerting and active response (which means that it can take some action automatically as soon as it faces a problematic situation).  A must-have tool in your toolbox, IMO.
  • Fail2ban.  Fail2ban is a small utility which does one thing, and does it good.  It monitors the system authentication logs, and detects password guessing attempts by counting the number of failed authentications, and bans the IPs of the offenders using the kernel firewall.
  • Monit.  Monit is a tool for managing services, processes, devices, files, etc.  Monit allows full customization by accepting flexible rules on what to monitor, and what to do in case something goes wrong.  It goes beyond simple stuff such as monitoring a port and pid file, and can for example send you an alert when a filesystem is about to run out of space, stop a service when it consumes too much CPU, or monitor an external server.
  • Munin.  Munin is an excellent utility which can collect data on just about anything going on in the server using RRDtool, and generate graphs for both online and offline viewing.  A few examples of the graphs it generates include network interface incoming/outgoing traffic, inode usage, swap usage, memory usage, and filesystem usage.  It also includes facilities to generate service-specific graphs, such as MySQL query count and Postfix mail queue size.

Posted in Blog Tagged with: ,