I read a lot about how other people create their servers.  Sometimes people ask what makes our servers so fast and so powerful.  I love our technology, but I don’t always have the words to describe it.  So I read what other people have written.

I found this article interesting.  It compares Node.js to older technologies.  The thesis is that Node.js is a nice product, but if you think it’s new you’ve bought into the hype.  The author pulled out a much older product which uses TCL instead of JavaScript.  They use the same basic tricks (epoll / asynchronous calls / one thread) and they offer very similar performance.  The older TCL code was slightly faster and could scale slightly better.

That article, and the comments people left, suggests that Node.js didn’t really invent anything.  That software just smoothed out a few rough edges.  They added libraries to make it slightly easier to write code for a server.  TCL and JavaScript were both originally used on the desktop.  They have powerful tools in them, but they weren’t customized for a server application.  The tools were always there.  Node.js just made the tools a little more accessible to the layman.

That paper referenced another.  In https://web.stanford.edu/~ouster/cgi-bin/papers/threads.pdf the creator of TCL discussed why a single threaded application is usually better than a multi-threaded program.  He discussed the tools required write code this way.  The interesting part of this second article is that it was written over 20 years ago!  The point is that only a few of us were doing this 20 years ago, but now it’s much more common.

Of course a lot has changed in 20 years.  One change is particularly relevant.  When that paper was written, most computers had only one one CPU.  I just ordered a new computer with 16 real cores and 16 more if you count HyperThreading.  And it wasn’t that expensive!  One of the big problems of our day is how to make efficient use of all of these CPUs.

At Trade-Ideas we actually do both of these things.  We have a front end like Node.js, or the TCL solution.  We often use just one thread (or a small number of threads) to talk with the users, and we use the same tricks described above.  We do this to keep things simple, so we can add new code when we need to.  But we also have real work to do.  (The first paper above was describing two web servers.  They spend most of their time copying data from one place to another, not actually processing data.  That’s what those tools are good at.)  So we manage a number of worker threads in a very efficient way.  And it’s all in one process.  The front end can send a pointer to one or more other threads to do the work.  You don’t have to copy any data from one application or one machine to the next.

We’ve developed a lot of our own tools to make this easier.  We don’t have to start from scratch each time.  Wether we’re in script or C++ we can quickly grab one of our proprietary libraries whenever we need to create a new server.  We only have to write code for the new features.  Our libraries take care of the details described above.  They are our libraries, so we treat them with care.  We are constantly smoothing out any rough spots, so they are easy to use.

It’s not about C++ vs scripting.  It’s not about one thread vs many.  It’s about picking the right tool at the right time.  It’s about making all the pieces work together, when you make different decisions in different parts of your project.  That is the magic at Trade-Ideas.