North American Network Operators Group

Date Prev | Date Next | Date Index | Thread Index | Author Index | Historical

Re: Scalability issues in the Internet routing system

  • From: william(at)elan.net
  • Date: Thu Oct 27 05:58:45 2005

On Thu, 27 Oct 2005 [email protected] wrote:

Another thing, it would be interesting to hear of any work on
breaking the "router code" into multiple threads.  Being able to
truly take advantage of multiple processors when receiving 2M updates
would be the cats pajamas.  Has anyone seen this?  I suppose MBGP
could be rather straightforward, as opposed to one big table, in a
multi-processor implementation.
Why bother multithreading when you can just use multiple
CPUs? :-)

Nowadays, a CPU is not a chip, it is a core. "Core" is
the name for a section of a chip which functions as
a CPU. Cores are actually software written in a language
such as VHDL (VHSIC Hardware Description Language).
VHSIC stands for Very High Speed Integrated Circuit.
The core is "compiled" into hardware on either an
FPGA (Field Programmable Gate Array) or an ASIC
(Application Specific Integrated Circuit). An FPGA
can be reconfigured by software at any time, for instance
you could reprogram an FPGA to do route lookups for
a specific set of prefixes and change the hardware
whenever the prefix list changes.

Most ASICs nowadays are actually hybrid chips because
they contain an FPGA section. Now, back to cores. Since
the CPU core is simply software, it is possible to
install multiple copies of the core on an FPGA or an
ASIC if there is enough space. The cores for RISC machines
like ARM are much smaller than the core for a Pentium
and therefore a simple RISC CPU core can be replicated
more times.

Now, with that information in hand, you will be able
to understand just what Cisco and IBM have done in creating
the CRS-1 chip with a minimum of 188 CPUs on the chip.
http://www.eet.com/showArticle.jhtml?articleID=26806315

Just as the line between routers and switches has become
blurred, so to has the line between hardware and software
become blurred.
Thank you very much for the above description - very interesting.
I do however note that multiple CPUs is not a replacement for multi-threading (in fact multi-threading takes advantage of multiple
CPUs very well).

When you add operating system (software) to your design of multi-cpu system on one chip substructure it looks like:

ASIC/FPGA
----------------------
| | | |
CPU1 CPU2 CPU3 CPU4
| | | |
OS OS OS OS
| | | |
Pr1 Pr2 Pr3 Pr4 (Pr is short for Process)

Which means the OS is replicated. In fact its even worse as the OS would have to be independent and use separate memory of fixed size.

In actuality when you do have multiple CPUs you still want the design like:

ASIC/FPGA
----------------------
| | | |
CPU1 CPU2 CPU3 CPU4
| | | |
----------------------
OS
----------------------
| |
process1-- process2--
| | | |
thread1 thread2 thread3 thread4

In above typical system design the OS is shared code for entire system that controls what actual code is run on what CPU (and the OS functions
are not replicated for each CPU/system) and the system can share physical
memory easily. Each process is independent piece of code where as thread
is multiple instances of the same process run simultaneously to take
care of particular operation (the separation into process and thread is
so that the actual programming code is not replicated in memory for
each instance of process).

And this system would still take advantage of multiple CPUs properly because OS make it so that each thread actually runs on its different
CPU in parallel - but memory management and programing code is shared
and not replicated as when you have fully independent parallel systems.

--
William Leibzon
Elan Networks
[email protected]