How FB-DIMM Memories Work

Introduction

FB-DIMM (Fully Buffered DIMM) is a memory module technology targeted to servers developed recently created in order to increase the memory speed and the maximum memory capacity of a server. In this tutorial we will explain what are the main features of FB-DIMM modules and the difference between DIMM and FB-DIMM modules.

The main difference between FB-DIMM modules and regular DIMM modules is that on FB-DIMM the communication between the memory controller (chipset) and the module is serial, in the same way that occurs with PCI Express, while on standard DIMM modules this communication is parallel.

Using serial communication the number of wires needed to connect the chipset to the memory module is lower and also allows the creating of more memory channels, what increases memory performance. With FB-DIMM technology it is possible to have up to eight modules per channel and up to six memory channels. So this technology increases both memory capacity and speed.

Each extra memory channel that is added to the system increases the memory subsystem transfer rate. For example, if you use a single DDR2-533 channel, the memory transfer rate will be of 4,264 MB/s. If you use two DDR2-533 channels, as it happens on dual channel configuration available today, the memory transfer rate will be of 8,528 MB/s. With four channels, the memory transfer rate jumps to 17,056 MB/s, and so on.

FB-DIMM serial communication uses 10 pairs of wires between the chipset and the memory sockets and 12 or 14 pairs of wires between the memory sockets and the chipset. Each pair of wires use differential transmission, i.e. the signal is transmitted on a wire and the same signal but inverted is transmitted on the other wire from the pair, using the same idea used on twisted pair networking cables. This is done to ensure that the receiver is getting the data correctly. Read this tutorial to understand how this kind of transmission works.

Another important aspect of FB-DIMM is that it uses different paths for data transmission and data reception. Standard DIMM modules use the same path to both transmit and receive data. The system used on FB-DIMM modules helps to increase the performance of the memory subsystem.


click to enlarge
Figure 1: How FB-DIMM modules work.

Initially FB-DIMM modules will use DDR2 memory, migrating to DDR3 when this kind of memory becomes available. It is very important to notice that FB-DIMM modules will have the same physical size as DDR2-DIMM modules and will keep this same size when DDR3 memories are released.

Motherboard Design

There are several advantages in using FB-DIMM memories on servers. We have already mentioned two of them: greater memory capacity and higher performance. There is also another great advantage that is the simplification in motherboard design, since the path between the chipset and the memory sockets uses less wires.

Even tough FB-DIMM uses standard DDR2-DIMM sockets, which have 240 pins, it actually uses only 69 of these pins, making it easier to make the motherboard routing (design of the wires on the printed circuit board).

On Figure 2 you can see the necessary routing to connect the north bridge to the DDR2 sockets using just one memory channel. A three-layer printed circuit board is needed.


click to enlarge
Figure 2: DDR2-DIMM routing, one channel.

Now on Figure 3 you can see the necessary routing to connect the north bridge to the FB-DIMM sockets using two channels. It not only uses fewer wires but also uses only two layers from the printed circuit board, making the motherboard simpler to design (the motherboard still needs to use a three-layer PCB; the third layer is used for power wires). Notice FB-DIMM using two channels uses fewer wires than DDR2-DIMM using just one channel.


click to enlarge
Figure 3: FB-DIMM routing, two channels.

Capacity and Performance

On Figure 4 you can find a summary regarding FB-DIMM capacity and performance advantages.

Using regular DDR2 memory modules the maximum memory subsystem you can build is of 8 GB and 10 GB/s transfer rate, if you use four DDR2-DIMM sockets, two channels and DDR2-800 memories.

With FB-DIMM, the maximum capacity jumps to 192 GB and the transfer rate jumps to 40 GB/s, if you use 48 modules, six channels, eight modules per channel, using the same memories (DDR2-800).


click to enlarge
Figure 4: DDR2-DIMM vs. FB-DIMM capacity comparison.

FB-DIMM idea is really interesting and should solve the demand for high RAM capacity servers, especially if we think that new CPUs from both Intel and AMD can access far over 4 GB of RAM when using 64-bit operating systems. So far several memory manufacturers have already announced that they will manufacture FB-DIMM parts, including Samsung, Elpida, Infineon, Micron, Hynix, Nanya and Kingston.

By Gabriel Torres on December 23, 2005

Originally at http://www.hardwaresecrets.com/article/266

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: