Back Online After Server Failure
This is exactly why I backup as frequently as I do. During this entire process no data was lost.Yesterday my webserver starting acting very strangely. I attempted to upload some photos to my photo gallery. It would not let me create any new content, but I could still view all of my content that was already posted. I SSH’d into the server and found that my system load was at 280 (!!!) and growing by about 2 points per minute. I checked some logs and couldn’t really figure out what was going on. No process was showing a major increase in resource consumption and there were no signs of extra traffic. I tried to close out all of the terminal windows I had open, but two of them would not lose, no matter what I tried (killing processes, etc.). After 225 days of uptime, I figured I should just reboot the system.
I waited a few minutes but the system never came back online (not even enough for me to ping it). I waited untill about 10 minutes after I gave the shutdown command and then forcefully turned it off. I brought it into my room to work on it and plugged it into a monitor to see where the problem was (I usually run my servers headless). I turned on the power and… Read more…