Jump to content

cbox


The 29 hours of downtime.


Recommended Posts

Posted

I'm going to run you through what happened.

whoa i don't like to read, skip to the end.

 

 

2 days ago right when I launched this new server it was running incredibly fast. It had a very light weight control panel known as Kloxo. It basically had only the necessities to run a web server, with the addition of security benefits. The site was running incredibly fast, and I finally felt that I had tweak the server perfectly. I'm sitting in SSH, checking over some configurations and I got an email saying there was a SSH login. Which was strange, since I'm the only user. I then get force booted from my SSH session and realize something was up, I could not log back in and I had 0 access to my own server. About 30 minutes later I received an email from some guy saying that Kloxo is worst piece of security ever and said he could take this entire server. His login was jailed to a tiny little directory though, so I didn't think he could do any damage. It was down to asking the DC for a reformat, or arguing with this random guy. 5 minutes later he emailed me again and said in a nutshell, get rid of Kloxo and put a real control panel on there.

 

This is all going on as IBM is running smooth as silk. After he sent me my new password, I quickly implemented a private/public key for log in, so now you need to steal my public key to get into SSH. Point being, I looked at the bash_history (what someone types into ssh) and surprised this guy did nothing except taunt me. So now I'm sitting with a free control panel and being forced to upgrade to my personal choice of cPanel. It was 20/month extra with my provider. I said hell no, and bought a direct VPS license from cpanel for $200, but wait. $20 x 12 months = $240, no I did save money :p

 

After I purchased that license I knocked the sever offline at around 3pm. I locked the databases to prevent any extra data and began to tar (like RAR) my entire user folder. About 30 minutes later I had all my dB's and files in a .tar.gz file. I then slowly but surely transferred that back to my computer. Took about 45 minutes to transfer. So now I had all my files and reformatted the server to a base 5.4 Centos install. I then began the cPanel install.

 

50 minutes later at the end of the install, I get this.

 

Securing the /tmp and /var/tmp directories....*** Notice *** No loop module detected
If the loopback block device is built as a module, try running `modprobe loop` as root via ssh and running this script again.
If the loopback block device is built into the kernel itself, you can ignore this message.
*** Notice *** No working loopback device files found. Try running  `modprobe loop` as root via ssh and running this script again.

 

I googled and could not figure that out for at least 45 more minutes. I tried reformatting to minimal Centos install on 64 bit, 32 bit, etc. I also tried all those suggestions online, and still no go. I then reformatted for the last time and asked cPanel to install the panel. Took them around 45 minutes to start the install, and then 3 hours to do it. By the looks of the bash_history they encountered the same error, but wrote some strange terminal code that I don't even understand to resolve it.

 

So now I'm faced with the initial control panel of WHM and cPanel. Servers are usually setup in days, and the tweaked over weeks. I think I spent 3 hours straight in that stupid WHM area just setting it up. Compared to Kloxo there was 1,903,912 different settings to pick from. All the way from pure-ftp or pro-ftpd, to the minor version of PHP. Once everything was setup I began the slow transfer of this massive 8ish GB tar.gz file. At 123kbps....

 

Granted, I figured out why my connection was being throttled and got it back up to 1mbps later on.

 

As this slow file is transferring I am performing my optimizations that I saved from our previous server. However, I learned that you cannot just edit my.cnf, or httpd.conf as cPanel will re-generate it with its version. You must edit another file, and save it. Which then modifies the config files. Took me awhile to get a hang of that syntax, as I was continually editing the "cached" version of files. I also was setting up the domains, fixing the DNS and adding the webmails for each of the sites.

 

2.5 hours later the transfer is almost done. That stupid large piece of junk tar file finishes. I simple untar it at command line and my entire folder structure is retained (in the wrong place), so I had to do a little cp -a (copy) to get that bugger back to where I wanted. Then came the problem of permissions and chmodding. None of my chmod's were saved, so I had to restore my permissions to allow files to run. Might have noticed those 403 errors?

 

Then the worst part comes, apache and virtualhosts with the www extension were failing. For at least 2 hours www.ibotmodz.net would go to a setup page, as ibotmodz.net would go to the site. I broke things and fixed other things for 2 hours until that was fixed. Databases were imported, permissions were restored and alas IBM loaded slowly. My server was back to its un-optimized self due to all the things I did trying to fix the virtualhost bug.

 

So I then I went to sleep and left IBM running un-optimize. Woke up a few hours later and began tweaking it to run a bit better. So....if anyone wants to chip in a couple bucks for the cPanel license. You know my paypal :p

 

 

 

TL;DR = There were problems. They were fixed. bi.

Posted

AIs can juggle ping pong balls.

 

I'm sure they can use captchas.

lol saw that video. But can they annoy the crap out of someone while they try to play an awesome video game?

 

Oh wait.... yeah they can....

http://images.wikia.com/halo/images/e/e4/Cortana-H3.png

Posted (edited)

@Peaches and any other AI

 

011000110110010000100000011100100110111101101111011101000000110100001010011000110110100001101101011011110110010000100000001101110011011100110111001

000000111001001101111011011110111010000001101000010100111001001101101011001000110100101110010001000000111001001101111011011110111010000001101000010

100110001101101100011001010110000101110010000011010000101001100101011000110110100001101111001000000010001001000111011011110110111101100100011000100

111100101100101001000000011101000101001001000100000110100001010011100100110010101100010011011110110111101110100000011010000101001101100011011110110

110001101111011011000110111101101100011011110110110001101111011011000110111101101100011011110110110001101111011011000110111101101100011011110110110

001101111011011000110111101101100000011010000101001110100011010000110000101110100001000000110001101101111011001000110010100100000011101110110111101

110101011011000110010001101110001001110111010000100000011001010111011001100101011100100010000001110111011011110111001001101011001000000011101001010

000

Edited by Quinn
Posted

I chmodded them all to a safe powerless number, then manually went back and changed those that wanted 777, and 750 and 444 etc.

its not that big of a pain in the ass, but it does take awhile.

Guest
This topic is now closed to further replies.
×
×
  • Create New...