A $622,000 computer and approaching the technological singularity

22,000 Supermicro 7089P-TR4T Barebone 7U SuperServer
80,000 8 x xeon 8180
440,000 96 x M386AAK40B40-CWD
80,000 8 x Tesla V100

12TB Ram
224 CPU cores / 448 threads
40,960 CUDA Cores
5,120 Tensor Cores
15.5 TFlops DP

$2776 per VM
224 VMs | 1 Core / 53GB Ram

$1388 per VM
448 VMs | 1 Thread / 26.5GB Ram


Ram is still stupid expensive.
Getting really close to 32 core (intel) cpu's.
Don't feel like digging into this boxes PCI express support (claims 39 slots)
Single xeon 8180 supports 48 lanes so in theory you could have 24 x 16 PCI cards with 8 cpu.
Some mix of GPU/Network/Storage controllers, you could just dump in all into network and put the GPU and Storage elsewhere.

technological singularity.. we're close

Things are gonna get weird, the system above is about 10^12 and is using 14nm process, 5nm has been reached by IBM (last summer) and we might get to 2-3nm.

We can't really make them run faster (heat) and we can't make them much smaller (quantum uncertainties) so we are scaling cores, dies are getting huge.

The last single core Intel was the Celeron 445 in 2013 (1.86GHz)

The current fastest single thread speed is the intel i7-8700k @3.7Ghz (6 cores/12 threads).

This is what a 28 core 8180 package looks like.

Moore's law

Current research is saying it will fail around 2025, but then you can't predict the future and people keep saying we can't keep following moore's law but then something changes and we can.

We are learning how to make things taller with 3d stacking.

This is IBM 5nm process with stacked layers

This was the intel 4004 circa 1971, it had 2,250 transistors and used a 10,000nm process (10um) and did 92,000 instructions per second on a 12mm² die.

Intel claims 100Bil per mm² with 10nm so that would work out to 1200 billion transistors on the die size of a 4004.. 533,333 x improvement in ~ 47 years.

The 3d part is interesting because we are just starting that path really.

This was sim city in 1989.

This is what people do in cities skyline (current city builder game)

This is Tokyo

You might see where I'm going with this, we might not be able to make chips smaller or denser but we sure can make them taller and we are just beginning that trend. I imagine cpu's in the years to come will have a surface complexity that resembles a city.

There are also quantum computers, this stuff is real, it's now.

An ordinary bit can be either a 1 or a 0, a qubit can be both a 1 and 0 at the same time.
Two ordinary bits have 2² potential positions.
Two qubits have that same potential all at once.
Adding qubits expands potential exponentially.

Google says "The ability of quantum machines to outperform classical computers is called quantum supremacy"

"a quantum computer with just 50 qubits could outperform even the world’s most powerful supercomputers"

IBM has 50-qubit quantum computer, quantum state is preserved for 90 microseconds.

"We are really proud of this; its a big frickin deal"

IBM says it has simulated a 56-qubit computer.

So we are right on the edge of making computer systems that can out think humans, all of them at once, this will lead to machines that are far more complex than humans can understand.

Interesting times..

Chaos and consciousness

Things to think about.

How do you you teach yourself that you are you at a young enough age to keep yourself from hurting yourself?
You are me, when they say we are one this is what this means to me.
We all kinda think the same because we are the same, but we are different.
Long ago something happened like the Big Bang, it ties us together.
We are all shooting through space in close to the same path but because of all the randomness of time we all have these differences.
We are echoes of an infinite past, something along those lines.
We are the spaces between it all, spirals and spinning, and physics.
When the Big Bang happened it was like ringing a bell, everything in the universe has a relationship via spin.
From day one, it's so complex we can't even make sense of it, but it's there.
It looks like a fractal, an oscillator, the universe is an oscillator.
It keeps exploding and collapsing on itself and who knows how long that period is.
We might not collapse either, just dissipate into space, but then the same thing happens again, pop here we go!
We have lived infinite lives, over and over again.
Each and every interaction adding to the equation, adding to the memory of it all.
Things like Star Trek and our wildest imaginations happened, but they are like memories from the past, or echos of the future.
Every time things collapse and explode you kinda stay in the same trajectory.
Why we feel so closely related, why everything around us kinda just works, it's how this part of space works.
If you go somewhere else it's different, there is probably math in there that could explain it.
Every time the Big Bang happens we go the same way, ish.
We have been down the same path infinite times but basically the universe is all the different ways you can spin things.
We all came from the same point in space.
Maybe we figured something out, and popped in here, it's just taken a few billion years to kinda get our shit together again.
Time may have nothing to do with it and it is just fooling us, are we are really just everywhere all the time?
Why do the ravens come sit with me on the playa and watch the sunrise?
Why do we all kind of speak the same language but are physically separated?
Why do we all share these strange ideas of god?
Distant memories of ourselves, we keep figuring it out, we keep leaning this but eventually it all ends and we start over.
Maybe if we can hold society together long enough we can figure out how to last, but how long is long enough?
The ideas of some of the sci fi stories stretch eons.
At the root of it, we are all just a mass of spinning energy that somehow has consciousness, we all vibrate the same.
Why does color and sound resonate so well with us, the periodic table?
It's all the same seed in a random number generator, illuminated by the same light.
When you add up all the random ways shit can explode and bounce from the Big Bang you get us.
But that's just this spot, Déjà vu.
All the weird shit we can possibly imagine has probably happened.
That is why we are so traumatized, but we can't really remember.
Why we feel love and compassion and empathy.
We have art and beauty too, we have everything possible, we have been everything.
look at that

$10.5mil 1260kW 9139.2 TF / 1827.84 DP TF (60TB ram)

one of these:
plus eight of these:
= 21.760 TFlops (single precission) / 4.352 DP TFlops in 4 u

380 gpu400-tx01 is unknown cost say $1500 for the chassis and backplane

2x x5570 cpu is $1462

18 x dd3 8gb is $1584 (144GB)

8 x v9800 $20,000

4u 21.76 TFlops $25,000 ~3000w

1 rack $250,000 30kw 217.60 TFlops

42 racks $10.5mil 1260kW 9139.2 TF / 1827.84 DP TF (60TB ram)
4.8million stream processors

if only..

48 cores, 512GB ram, 16GB video
10TB Storage (1.7 GByte/sec, 3 µsec latency, 1 Million random or sequential IOPS)

$1927 B8812F48W8HR 4U TYAN S8812 (S8812WGM3NR)
$1199 x4 ($4796) AMD OPTERON 6176 2.3gh
$553 x32 ($17,696) Kingston ValueRAM 16 GB Memory - DIMM 240-pin - 1066 MHz ( PC3-8500 )
$15,117 PNY nVidia Quadro Plex 2200 S4 Graphics System
$621 HP NC550SFP Dual Port 10GbE Server Adapter Network adapter
~$200,000 10TB Violin 3200
$3,998 (used) ViewSonic VP 2290b 3840x2400


could be expanded to 64 cores when the 16 core opterons are out.
could double the throughput on the storage with another v3200. +$200k
a second Quadro Plex 2200 S4 could be added.