What tool do you recommend to track changes on a Linux/Unix server.

I manage several Linux servers for clients in several roles like email, caching, web serving, filtering, firewalling/routing, and so on.

Since I don't own these computers and just provide remote support, central management systems like Puppet don't seem like they are the correct tool. (Please correct me if you think I am wrong about this assumption)

What tools do you recommend to track changes of configuration files, package installs and so on?

I am thinking something like etckeeper may be near to what I need, but I want to know if there is something better.

---

Update

We will have backups of the systems, and I wouldn't expect this type of a tool to be an alternative to a backup. This is about keeping track of changes of configuration and having a system to know what changed when, by who, and hopefully why.

From serverfault Zoredache

• You may want to look at Tripwire or AIDE

  Both will track config file changes on your machines.

  Christopher Cashell : Tripwire and AIDE are really good tools, but they're both *very* heavyweight compared to changetrack or etckeeper. If you need full integrity checking and host based IDS features, they're a good choice. But if you just want to monitor changes made by other admins, I wouldn't recommend them.

  Mark Turner : I would agree with your assessment if it were only about Tripwire.. but AIDE is extremely easy to use for tracking just filesystem changes.

  Christopher Cashell : It's been a while since I've played with AIDE or Tripwire, but when I was looking for something to monitor config changes (I eventually went with Changetrack), my playing with AIDE suggested it was difficult or impossible to get it to simply send diffs of any files changed in the last hour, send every hour (for example). That, plus storing the changes in some sort of revision control system was all I needed, and AIDE seemed cumbersome for it. That was some years ago, however, so things definitely may have changed.
  From Mark Turner

• I've looked at etckeeper, but I haven't used it. However, I have used Changetrack. I've been using it on all of my home machines for many years, and at my previous job it was part of our standard server install. We used it there for the last five years, and had it installed on about 200 boxes.

  The setup is trivial (I created an RPM for it at my last job), and the configuration is really simple. I generally set it up to monitor all of /etc/.

  gbjbaanb : so why havn't you uploaded the rpm to the changetrack project :)

  Christopher Cashell : I actually planned on doing just that. However, I ended up leaving my last job before I got around to uploading it, and no longer have access to it. I'll ping some of my former co-workers, though, see if they can do it.
  From Christopher Cashell

• I've got etckeeper on my personal workstation, but I've not had to do much with it yet (other than have it track all my changes). Seems like it does a reasonable job of making sure you at least know what's been fiddled with.

  I wouldn't write off Puppet as a solution -- as long as some of the services on the machine are your responsibility to maintain, then a system that makes sure that if someone jiggles your config that it gets put back the way you want it is a godsend.

  On the other hand, if others make changes regularly (and they don't usually screw it up), you might have to resort to just tracking what other people have done for later disaster recovery. Don't forget that things will be changed all over the place, so a full-machine checkpoint tool might be better. I'd perhaps even consider going full-disk incremental backup on it (like rdiff-backup or something) to be sure you're tracking everything (maybe drop /home and other user-level areas out of the backup, if you just want to track administrative changes).

  From womble

• A simple file monitoring script is filemon .I use it on my home PC,and combined with crontab it does a simple and easy job . For a more complex solution of integrity check (file changes,new packages installed and many many other) I use OSSEC on a bunch of servers .

  From Dan

• For tracking package changes (installs, upgrades, etc) on RPM-based systems, as long as all changes are done with yum or yumex, each package change is logged in /var/log/yum.log.

  Other people have already answered tracking changes in /etc. Don't forget that you also want to track configuration changes to bind which are partially in /var (at least on many Linux distributions) and that web pages are under /var/www on many Linux distributions. There will be directories outside /etc that have important configuration information on them.

  Depending on how things are managed, you may also want to track /usr/local/etc and other directory trees (/opt, some trees under /var, and anything else that is specific to your customers).

  From Eddie

• You could put /etc under a dvcs such as git. You would commit every time you make changes and then you could just git diff whenever you started a job and be shown all the changes.

  Zoredache : Good idea, this is exactly what etckeeper does. It supports git, mercurial, or bazaar. It connects into your package management system to keep track of changes related to packages adds/updates.
  From docgnome

• LBackup has support for logging of file deletion, modification and addition.

  From lucidsystems