And then it failed again this weekend, and this time I thought ahead a bit and tried a probe-ide at the console. No hard drive. It just forgets that anything's connected to the first IDE interface. Alright, that explains why it fails the way it does. Came back after a power cycle again, but now it was bugging me, so I started digging through syslog to get some idea of the timing.
So now I know that I have a machine which hangs solid at 4:27 PM on Saturday. Every week. The scheduled reboot happens after that so wasn't happening at all. There's nothing in anyone's crontab at 4:27. It shares a rack with a handful of other boxes, but nothing that requires weekend intervention like a tape drive. The other U10s in that rack are unmolested. It's very underloaded, and there's no significant mail traffic around that time.
What the hell?
