I present to you one of the projects that I have been working on for the past year. When I moved in and started fixing up my (new to me) house I found a 1950’s coke machine left out in the back yard (There was a lot of random junk out there). It took me a while to figure out what to do with it, but once I decided I wanted a video arcade room in my house, I knew exactly what to do. It turned out to be the perfect size to fit a custom tailored server rack! Below you will find pictures of the build process. You could always skip to the bottom to see the finished product, but where’s the fun in that?

The machine looks like it was brought to the yard on a wooden skid… that had in the years since rotted out. This caused the machine to slowly rock forward and have the door open. The open door let decades of weather destroy the interior. Luckily for our project we didn’t really care too much about that.

We figured that the best way to get the center drum out was to lay  the machine on its back. The drum was designed to hold 250 glass bottles and is, like the rest of the machine, made with solid steel.

Reopening the casket.

The center drum was held in place by a combination of gravity and pressure. We used wooden boards to pry the drum out. It was heavy, but detaching it was easier than we thought. Actually lifting it over the lip was more problematic.

Luckily that can be fixed with a time lapse!

We had a bit of a… gravity error. The slippery snow didn’t help matters. It turns out the balance of the machine gets really weird when you remove more than half of the insides. Luckily there was no damage to either of us or the machine.

Because of the snow and other debris in the back yard we had to go the long way around the house. I decided to take a picture of it while it was on the driveway.

Prepairing to bring it up the stairs to the "Workshop"

We started the modifications by drilling a series of pilot holes and then using a saws-all to cut the back of the case off. Once we got one of the sides done we started running into an issue with vibration. There was actually an inch or so of fiberglass insulation sandwiched between two layers of sheet metal. This caused them to shake more and more wildly the further John got. To rectify the problem I put on some heavy leather gloves and held the sheets in place as he cut.

After the back was cut off John and I took turns grinding down the lightly rusted metal on the inside of the machine. Our lithium-ion batteries for our drill certainly got a work out. Both John and I have a matching set of Milwaukee 28V tools, so we were able to have two batteries charging at a time. As soon as one of us drained the battery in the drill we could swap it out for a fresh one and have the other person keep working.

I found a used APC server rack on craigslist for just under $200, so John and I took my pickup truck over and loaded it up. It was actually in really good condition so I felt a little bad about the fact that I was about to cut it up.

Once we got the rack back to the house we started dissembling it. John unscrewed the front while I unscrewed the back and it no time we had all of the extra parts stripped out.

The server rack and the coke machine meet for the first time.

Before we cut the server rack down to size John and I took turns welding steel reinforcement bars onto the rack. This was our first real welding project so we practiced for a couple of hours before hand. The scrap pieces from the coke machine made for great practice. We used a small arc welder from Harbor Freight that plugged in to a 120V 15A outlet. Unfortunately the electrical in my house hadn’t been finished yet, so we had to run an extension cord from the boiler room.

After the reinforcements were in place John cut the top of the rack off. It makes a really cool coffee table!

The above picture is of the first test fit. We had some issues getting the top of the rack through the space we cut, so John took another couple fractions of an inch off the rack. It’s snug, but it works well. Note that this is one of the few times we loaded the rack through the rear of the machine (instead of through the door on the front).

The insulation and jagged metal edges needed to be protected so I cut strips from what used to be the back of the machine. John then drilled holes and screwed the pieces together. It was still too sharp for our tastes, so we went back to Home Depot and got some L pieces.

We couldn’t find exactly what we wanted, but these ones worked out pretty well. The L pieces are kind of thin, so they tended to warp slightly as John mounted them. After some work he got them looking how he wanted them.

Once John got the L covers in place I set to work prepping the inside of the Coke machine for new paint. In hindsight we should have painted the inside first and then mounted the shiny L pieces. Oh well. I just had to be a little more careful as I sprayed paint on. I used a paint sprayer powered by a separate air compressor to blow the paint on. I think it looks a lot better than it would have if I had tried to use roll on paint or spray cans.

After the grey primer was dry I put on two coats of black paint. I used Rust-olieum paints throughout the project.

I should mention that it was rather cold while we were doing all of this work. We were in the process of rebuilding my house after striping it down to the bare frame. I used a kerosene heater to help the paint dry (and also keep us a little warmer, I suppose). It was probably in the mid 30’s (‘F) at the time.

I put two coats of black paint on the rack. It took a couple of cycles of paint-dry-flip it over-missed a spot-repeat before I finally got everything. Server racks have a lot of spaces that are difficult to get to and see, but I wanted to make sure I got everything, not just the parts that would be visible.

After I finished painting the rack I started on the exterior of the Coke Machine. Once again I began the process of swapping out battery after battery in the drill. A corded drill may have been more practical from that perspective, but the cord would have been really annoying for working on the top of the machine, and besides, we had two sets of power tools and batteries to work with

Candy Apple Red

Here you can see the contrast between one partial coat of fresh paint and the original, sanded down, old paint. The paint that I found was a bit brighter than what I was aiming for, but it was the only one from Rustolium that was even close. It turned out to work wonderfully, even if it isn’t exactly an “official” Coca-Cola color.

The above picture is after one full coat of new paint. After it dried I put another full coat and then an additional set of touch-up spots.

I left the detail work to my friend Trey and my girlfriend Christina while I worked on wiring up some Ethernet runs through the house. They took their time and did an amazing job. All of Trey’s experience painting Warhammer miniatures paid off

The final paint job, before removing the painter’s tape.

After all of the paint was dry I reattached the chrome pieces. The bottle opener still works!

My friend James helped us move the rack into the arcade and then slide the cage in place. From this angle you can see one of the braces (top right of the rack) that John welded to the rack to help hold it in place. The braces make sure the rack stays stationary but also prevents the rack from sliding in from the back side.

John posing in front of the newly installed server rack.

It turns out that the floor was kind of warped, so we used plywood to prop the front of the rack up and make it level. It’s hard to tell, but the coke machine is actually slid into an old doorway that leads to the boiler room. I picked this location because the servers will suck in cool air from the living area of the house and blow out the hot exhaust freely into the boiler room. After several months running in this configuration the only heat-related issue was from a small fanless fit-PC that was sitting on top of the router. I moved it to its own shelf and didn’t have any more problems.

All of my gear mounted to the rack

On the front of my rack I have  (from top to bottom) a 24 port gigabit switch, an 80 port Fast Ethernet Cisco Switch,  1U Firewall, 2U KVM (with two user outputs. One for the rack and one to go to my desk), 1U backup router,  4U backup fileserver, Monitor, 4U Primary fileserver, and finally a 1U Webserver. Below the rack, where the compressor used to be, you can find the APC UPS that I’m using for now. The load is fairly high on it, so I’ll be getting second one soon.

From the back of the rack you can see the centralized CAT5e run, 96 port patch panel, and external 8-bay hard drive enclosure. All said, I have 32 TB of raw hard drive space connected. Less than half of that us useable due to backups and redundancy.

Since I had all of the walls taken apart for re-doing the house’s electrical system I took the opportunity to install centralized CAT5e throughout. I ran everything through 1/2 or 3/4 inch tubing so that I could replace it later if I decided to upgrade to CAT6A or fiber or anything else. In the above picture you can see the FiOS ONT as well as the dedicated 30A lines (one is 120V and one is 240V) that are run to the rack.

The entire house is connected to a generator (currently 3.5KW gasoline but I plan to upgrade to at least a 10KW diesel) as well as my military truck which has a 3.5KW inverter from Outback. This allowed me to keep the house (and computers) up and running when we had a storm come through that took out power for two days.

I would like to thank John for his extensive help with the project and also Trey, Christina, and James for their help as well.

Here’s the answer to the tricky part:

Fuse is a little special (I’m sure for good reason) in that it doesn’t like other people running it. Whenever I would mount the remote directory via SSHFS it would work fine for the virtual machine, but I could never successfully share it out via Samba. It turns out that Fuse wasn’t letting Samba gain access because Samba runs as a different user than the user that mounted the file system.

The fix:

Edit /etc/fuse.conf so that the line “user_allow_other” is uncommented (or, if the file isn’t there, touch it and then add the line).

Then mount the SSHFS to the directory that you are sharing via Samba. The key part is that you must use the “allow_other” option.

For example: $ sshfs example.com:/media/alpha /media/remotessh/ -o allow_other

That fixed it for me. If that still doesn’t work, double check your samba config file to make sure the permissions, passwords, etc are all configured properly and that your user account is added to the “fuse” group. For reference: My server is running Ubuntu 10.10 and my client VM is running Ubuntu 11.04. Good luck!

http://www.wynia.org/ was a great help for me figuring this out.

Earlier this week I had a motherboard die on my primary file server. This was an inconvenience, but not of terrible concern because of my (tested) backup and recovery plan. Throughout the entire ordeal, I didn’t lose even a single file. I did, however face a few interesting challenges.

I kept a server on the same rack ready to go as a cold spare. To use it I would simply disconnect the hard drives from one system and plug them in to the new system. All of the software was already configured. This was a good plan, except that in the year since I put it in place the SATA add-on card had died, so the spare system wasn’t useable. Lesson learned: Check your systems at least every now and then, not just once.

I ended up merging the two systems together, but that had its own set of issues. The biggest of which was my spare motherboard didn’t have an integrated graphics card, so its 16x slot was in use with one and I resorted to PCIE 1x and PCI cards to fill the need for SATA ports.

In order to use my SAS card from my primary machine I would normally have had to purchase a new motherboard. Instead, I remembered that I had read that the PCIE specification allows cards to run slower than they are rated (I.E. with only 8 out of 16 lanes active). I also remembered seeing someone who had taken a dremel to their motherboard and cut of the plastic card at the end in order to use a 4x card in a 1x port.

After some Googling I found the ribbon pictured above. It’s a little fragile, but it does the job. I ended up mounting it to a piece of particle board and screwing that into my case with a decking screw (They’re the same diameter as the regular case screws. Go figure). Total cost was $4.38 with Amazon Prime shipping (which I upped to 1 day). The system is working very well now.

Because of this experience I have revised my backup and recovery plan. All secondary arrays are now contained in their own USB (Version 2 and 3) cases. This means that I can connect them to any Linux computer and access all of the files. This also makes things easier to grab in the event of a fire (yank and run, instead of unscrew, left out, and waddle).

A total of 24 Connected Hard drives.

After a long time without updates, I am now getting to the point where I am comfortable with the idea of posting detailed(ish) updates again. I’ve been working with my friends for the past year on the 6800 project and because of this have not had much time to work on my truck. I thought I would post this quick update to show that, yes, things are still moving forward. The truck now has a space between my shed and the house, as well as having its own 20 amp electrical circuit.

As you may have noticed, I’ve been rather quiet over the past several months. That is because I’ve been spending the vast majority of my time on a somewhat unique project. I haven’t mentioned this project much on the public internet because I didn’t want to start the project with extremely high expectations and then never post about the work being done. However, after starting this project at the end of May 2010 my friends and I have come a long way. My goals are in sight and progress is being made every day.

Rest assured that there will be write-ups and many photos from throughout the process. And yes, I am being very vague.

For now I’ll leave you with the above  image. My friends already know what is going on, and I will reveal details on the internet once the project is “Live.”

For my setup I am using software RAID/LVM and not the built in RAID abilities of the card. If you want to use the card instead, it will map the entire array to a single “hard drive” in linux, such as a 4TB RAID 5 array mapped only to /dev/sdb

This is the highpoint website for the series of cards: http://www.highpoint-tech.com/USA_new/series_rr2600.htm

Follow these commands:

$ mkdir highpoint-raid-stuff    #You can call the folder whatever you like
$ cd highpoint-raid-stuff/
$ wget http://www.support-highpoint-tech.com/Main/rr26xx/268x/Linux/opensrc/rr268x-linux-src-v1.4-091124-1558.tar.gz tar xzf rr268x-linux-src-v1.4-091124-1558.tar.gz
$ cd rr268x-linux-src-v1.4/product/rr2680/linux/
$ make
$ sudo make install
$ sudo modprobe rr2680

Next, create a file called rr2680.sh in the /etc/init.d folder

Edit the file so that it contains the following text, then save it.

#!/bin/sh
modprobe rr2680
exit 0

Then run the following command:

$ sudo ln rr2680.sh /etc/rc3.d/rr2680.sh

After that, the server should automatically load the new driver each time it boots and you should be able to access all of the drives connected to it. I hope this helped!

This is my broken speaker from my car. It never worked for me and it turns out that it was actually made with paper. I’m replacing my entire stereo system in my car and will post pictures once I have the new gear and installed.

We brought Setesh with us this year to the Jimmy Buffett concert in Bristow VA. We gave countless tours of the truck and had a blast! Everyone walking by stopped to take a look. We’ll be back again next year with even more things for the truck.

Bringing the deuce and a half to Jimmy Buffet this year was a huge success. It went better than we could of hoped for. Thank you to all the people who helped make this possible and to everyone who stopped by for a tour. It was a blast. I’ll post more pictures when I get around to it…

Talking after a game

Today we celebrated my brother’s birthday and the new tires for the truck by gaming in the Kingstowne parking lot. After some struggling with the inverter everything worked out perfectly.

The new tires look great

I’ll post more pictures in a day or two…