Need advice on a high-end server purchase

How many IOPS do you need? Enterprise grade SSDs or FusionIO cards will give you lots more IOPS then spinning disk.

Probably best to hit up a forum like "servethehome" that has people who actually know what that equipment is. What operating system? Do you have a handle on the sort of CPU, RAM, and I/O use of the application?

Do you know anyone in the universities, etc., that you could talk to/perhaps borrow use of their system to sort of evaluate what you need? Maybe a research 'accelerator' or something?

I'm thinking there's maybe $30-$40k of hardware there. So we're n

rilles wrote: ↑ How many IOPS do you need? Enterprise grade SSDs or FusionIO cards will give you lots more IOPS then spinning disk.

Good question. How much writing will you do? If it's not a lot (sounds like you will run in mem like HANA stuff) you can go entry/mainstream SSDs. Around 6 SSDs, anything more you will likely saturate your controller anyway. I would not recommend enterprise class SSDs as most people will not use them as they are meant and will pay more for no reason.

Spinning disks don't hold a candle to SSD when it comes to w/r performance, go RAID5 w/hot spare or RAID6 with the SSDs. Even like this you will have much higher performance than Raid10 with 15k drives (not to mention the heat and electricity). If you don't need high IOPs, and workload isn't random, use SATA drives, they are fine for sequential writes.

Sorry, can't help on price; all that stuff is now on its way out, so you should be able to get a good deal.

But back to your question for the OP, what IOPs do you need?

I really don't know enough to answer this question about IOPS...does the number 100-150 sound right?

For low IOPS just a few RAID1 consumer grade SATA SSDs would use way less power, make way less noise and would be less complex to manage and you can ditch the raid controller and use software RAID (1TB SSDs are getting real cheap now). But I guess this depends on the criticality of the service and the personal leanings of the server admin.

Can’t comment on the rest, but you don’t need a video card in the server to remote into it.

1. Apparently that CPU is equivalent to a Core i7 5960X. If several benchmarks on the web are to be believed, it's outperforming a Kaby Lake processor easily by 10-15%. So at least we know that no i5 will outmatch it persormance wise.

Power wise seems to be similar to Sky Lake/Kaby Lake.


2. RAM brand does not matter. Most of it is from the same suppliers anyway, even the ECC stuff.

3. 15k SAS drives "should" be decently fast, but will not beat even a cheap SSD. Those drives are higher performance compared to SATA, by usually 10-15%. Where SAS shines is reliability and seek times due to very fast spin speeds. For RAID, well, if all you care is about the quickest access to the data, RAID 0, but it offers ZERO protection for the data on your disks. If you set all disks as RAID 0, your server will see a large logical disk that is really fast, but one drive dies, and you're toast. RAID 0, and religious backups to another array, or tape, anything. RAID 10 is great for a little fault tolerance. You can usually afford to lose one disk and data still safe. But data storage is cut in half. 4 disks in RAID 10 is getting the performance of 2 disks in RAID 0 with a backup RAID 0 in parallel. RAID 5 sucks for writing to, but about equal to RAID 10 for reading. You need a very good controller for decent performance. You can afford to lose one disk, but then you're running in degraded mode (meaning slow as molasses) until disk is replaced. Only advantage of RAID 5 is that you only lose one disk in a set (If you have 6 disks, RAID 5 you lose one to the parity. RAID 10 would you lose 3).

Setting quality SSD's all on RAID 0 would yield some super kickass data access, and be "reasolably" safe from death. You could install the OS on the RAID 0 and the system would not suffer much in performance. Backup to other device CRITICAL though.


4. iDRAC. Do you need to access the Server remotely to monitor stats? or to Shutdown/Manage remotely?

5. Optical drive depends on your needs. Some stuff is still on CD/DVD. But most things you can load on USB stick now.

6. Redundant PSU usually means "each" PSU is 1100w, for a total of 2200w, but the System will not use more than 1100w total, so two PSU's share half the load each.

7. PCI-E server slots basically the same as desktop PCI-E. RAID is speficically for RAID card. NDC is a Dell proprietary Network Card with proprietary connector. And you can usually get riser cards that add 2 more PCI-E.

Limitations on cards you can install there usually depend if Dell's Hardware Compatibility List allows installs.

In my personal experience, you can usually install eSATA or USB 3.0 cards there, and as long as there's a Windows 7 or Win 10 driver, it will work in an Win Server OS (with a bunch of fighting/hacking Windows Server driver signing)

8. Absolutely do NOT need a GPU to access server remotely. You usually attach a keyboard/mouse (or KVM switch) to server to install/configure a Server OS (Win Server, ESx, Some flavour of Linux Server). Depending on what OS, that will determine remote access to it. Windows can use RDP or VNC (Linux is VNC primarily I believe). ESx will be ESx Client/VSphere, etc)

For computation it is possible to install some GPU's, but normally not officially supported by Dell or whatever company makes the Server. Most of the time they might not even physically fit, and at least Windows Server will give you a hell of a time to load drivers for it.

rilles wrote: ↑ For low IOPS just a few RAID1 consumer grade SATA SSDs would use way less power, make way less noise and would be less complex to manage and you can ditch the raid controller and use software RAID (1TB SSDs are getting real cheap now). But I guess this depends on the criticality of the service and the personal leanings of the server admin.

Most Servers come at least with a basic hardware RAID Controller. Dell uses the PERC series.

"Software" RAID can mean different things.

Example:

A PC might have the Intel RAID config utility in the BIOS. Usually only has RAID 0 or 1. All that is is a passthru for the Intel CPU to handle the RAID. RAID 0 and 1 have next to zero overhead for what most home users will do (maybe tops 6 HDD's in RAID?)

Once you get to Enterprise Level, with Storage Appliances that have 48+ disks, well at that point you start needing dedicated controllers.

The other "software" RAID is something like Windows Dynamic Disks. All that is, is that you boot to Windows, and whatever disks you have installed you can bunch them together and format them in a large "logical" disk, as RAID something. CPU usage also negligible for RAID 0 or 1. And with DynDisks you cannot install the OS to a DynDisk.

So what's the advantage? Far FAR easier to transfer those disks physically to another PC. Any Windows will recognize them as a RAID set.

If you do the "Intel" RAID utility thing, you risk not being able to read those disks on older Intel chipsets, or non Intel ones.

Dedicated RAID controllers? Usually you need the same controller to read, or you're (mostly) toast.

My understanding is when tasks are large enough to migrate to GPU, the whole thing goes and CPU no longer relevant i.e. no gain in splitting.

Then the bottleneck shifts to disk io (load/write back results) and the channel between GPU and RAM, not CPU or CPU cache.

Finite element analysis is suitable to highly parallel processing. Hence, you can use lots of GPU cores rather than CPU as the previous poster suggested. The biggest super-computers are GPU based and they are suited to big FEA problems.

It will also be processor bound, so disk access may not be critical. You will need a MB that can carry a lot of GPU cards and you'll be running a Linux cluster distro. Otherwise. several MBs that are clustered, each with several GPU cards. Think game computers rather than server.

Trivia: Years ago, the McLaren racing team did this for about $50,000 using 8 Playstations (Cell Broadband Engine Architecture) and a workstation as a controller. Sony killed the Linux capability and this was a dead end.

Thanks all for your help! In the end, I did purchase this server with a few modifications -- no HDDs (I bought 3 x 2TB Micron SSDs instead) and no graphics card. I also ensured that the RAM was Samsung and 1600MHz. It cost me CAD $14K.