Some doodlepad thoughts on lowcost storage for SMB.
Disk Shelves
- NetApp StoreVault. NAS/iSCSI included, FC extra. Snapshots, data replication (sorta-cluster). Base price, 1TB raw, about $5k. Fully loaded, 6TB raw, about $15k according to eWeek (with CIFS and NFS licenses). 6TB raw (12x 500GB disks) would translate to approx 3.3TB usable, assuming 1 hot spare drive, double parity protection, 20% snapshot storage, and 5% overhead for OS (6000 - 1500 * .95 * .80). This is worst-case; you can add back approx .7 TB by going to single parity protection and no hot spare. The eWeek article says they benchmarked at about 65mB/sec read, 43mB/sec write speeds.
- Using DFS and multiple Storevaults you could masically scale almost infinitely while still maintaining a cohesive storage namespace.
- IBM DS3300 is a blocklevel iSCSI device, not NAS. Running through a quoting excercise with a friend in the biz, we came up at about $18k for 4TB usable, assuming the customer has the requisite gigabit nics and switchports and interconnects. You don't get snapshots or quotas in this price.
- I don't know pricing on the IBM N3700 which is more comparable to the Storevault (NAS and iSCSI in one box).
- Xinit Storage Networking. Xinit are the company behind the OSS OpenFiler, which is smashing NAS/SAN prices. Their extensive feature set includes all the basics - CIFS, iSCSI, quotas, snapshots, integration with multiple enterprise directories. The project has had active development and support for years, and shows no signs of slowing down. However, as of today I was unable to get pricing from their store.
- PogoLinux have several offerrings based on StorageDirector 3008 can do about 7.5TB of usable storage space (16x 500GB SATA disks) for $11,000. There seems not to be much online documentation of their product; it does not look like a repackaged Openfiler distribution.
Networking (focus on copper not fiber)
- Gigabit Ethernet means 125 mB/sec max speed per port. 10 gigabit ethernet (10GBE) means 1250mB/sec
- We can basically rule out 10 gigabit ethernet (10GBE).
- Pricing for 10GBE switch ports is hard to find. But most likely still above $1000/port and therefore out of range.
- 10GBE iSCSI HBAs are on the horizon. Some are available, but you can't really say they're practical yet.
- Also note that copper 10GBE is likely to be short-haul only - 15 meters or less.
- iSCSI: To HBA or Not To HBA? That's the question. The article links to the interesting emBoot solution ($95 per server) for booting iSCSI without an iSCSI HBA.
- Qlogic iSCSI (gigabit, copper) HBAs start around $790. Alacritech has one in the $365 range.
- 24 port manageable gigabit copper switch from Dell, $770. That's about $33/port.
Quick pricing comparison. Assume we need 8 servers, each with 500GB of storage (including OS, data, apps). That's a total of 4TB (usable) space needed, and we'll spec it with both dual redundancy (RAID1) and no redundancy (single disk). We'll price as pizzabox rackmount servers and we'll assume SATA storage for all pricing excercises. The pricing was done via web on 10/10/2007. Eight servers, each with 2x local 500GB drives is $25,312, and 8 servers with no drives is $19,728. Our local DAS storage price, then, is about $5600 ($25312 - $19,728 = $5,584) if we want any redundancy. If we do without redundancy, we have a lower DAS price: $1680. Hard to beat!
We'd pay about $15,000 for a fully loaded Storevault that could provide us the same 4 terabytes of RAID storage we could have gotten as DAS in the Dell systems for about $5600. Therefore we'd be paying about $9400 (15000 - 5600) for the convenience of NAS, snapshots, and a better quota system. We'd pay a bit more for the IBM DS3300 series: $12,400 (18000 - 5600) above DAS cost. And with the Frankenbuild NAS (parts list and prices below) running OpenFiler, we'd actually save $1046 (5600 - 4554) over DAS cost, but we'd have no warrantee support of our storage beyond the individual parts we bought. Though we could engage Xinit to professionally support OpenFiler for an unknown cost (email them for info). Maybe a better way to go would be to build two Frankenbuild systems for about $9100 and keep them synchronized, in which case our cost would be $3500 (9100 - 5600) over DAS.
Is it too much? From what I see here, there's really no breakeven point where a commercial NAS/SAN array is 'free' (ie: hardware costs no more than DAS internal to each server would cost). Although it's possible to frankenbuild your own NAS/SAN array, I imagine a lot of enterprises would shy away from this. So if you go the commercial route, you've got to find value in the other operational benefits. And if you go the Frankenbuild route, I'd love to hear from you!
Manager-friendly graphic representation
The Servers
There's no major preference for Dell here; it's just that they offer fairly easy online pricing. The storage price differential would remain roughly the same no matter which server vendor we chose, though it might change a little depending on that vendors disk prices of the day. I've explored some blade server options (they are more expensive) on the BladeServers page.
Base Price: $2466
- Dell PowerEdge 2950 with single Quad Core Xeon 1.6Ghz proc - $1188
- 4GB RAM (4x 1gb sticks) adds $335
- Windows 2003 R2 (x32 or x64) adds $559
- Rack Rails add $174
- Redundant powersupply and Y-cord adds $209
- Dual Embedded Broadcom 10/100/1000 TOE card is included in base price.
- An 80gb SATA hard drive is included in base price
- CD drive included. Add $48 for CD/DVD drive (not added)
- No floppy included. Add $21 for a floppy drive (not added)
Diskless boot: $2561
Single SATA 500GB (no redundancy): $2676
- First 500GB SATA drive adds $210
RAID1, 2x SATA 500gGB drives: $3164
- First 500GB SATA drive adds $210
- Second 500GB SATA drive adds $279
- PERC 5/i 4, 6, or 8-slot RAID backplane adds $209
So for 8 servers we'd pay
- 19,728 with no hard drives
- 21,408 with 1x SATA 500GB drive each
- 25,312 with 2x SATA 500GB drives in RAID1
Frankenbuild NAS
quux's mexican jumping wiki 
Uploading ....