I hate unexplained phenomena...

[mdlbear]

... especially when they're unexplained warnings from my intrusion-detection script. Especially when they only happen once, and can't be repeated by rerunning the script in the morning. Had I been cracked again?

When I got in this morning, I had email waiting for me from chkrootkit running on tatooine, the dual-boot machine in the office that's been spending most of the week turned off. It warned about a shell history file (belonging to a long-unused account) linked to /dev/null. Quite properly, too. But why didn't any of the other copies of chkrootkit report it? And why couldn't tatooine's copy find it when I turned it on in the morning? Very troubling.

So the next thing I did was to look through chkrootkit, which is just a shell script, and determined that the test in question involved a simple find command. So I tried that (slightly simplified), and it found the link just fine. Then I ran md5sum on all the copies of find I could, er, find. All the same. Then I downloaded the most recent version of chkrootkit. Same result. Weird. So things stood when I went out for a long walk.

After about two miles, things finally sorted themselves out. I had spent yesterday evening installing and configuring internal firewalls, in particular on the main fileserver where the home directories live. It was a little before midnight that I powered up tatooine, installed its copy of shorewall, and went off to bed contented. What I didn't realize until this morning when I checked it was that the Linux version of NIS uses a different port (786) for the portmapper than NFS does (111). Running NIS through a firewall is, of course, completely undocumented. It's a major security hole, so apparently nobody in their right mind ever does it.

I never scored high on sanity. The consequence of the closed port was that, when chkrootkit ran on tatooine in the middle of the night, it couldn't get the most recent list of usernames, home directories, and login shells. The other thing I'd done last night was to disable a bunch of old users by setting their shells to /bin/false. Since tatooine hadn't heard the news, it used the old password database. With the now-disabled user still active.

This morning I found the problem with NIS, fixed it, and then read my effective but alarming email. Naturally, since tatooine could now contact the NIS server on nova, it didn't see the problem anymore. What problem?

So things were OK after all, and apart from making me waste all morning chasing my tail, no damage was done. But at least I get an interesting post out of it.