Saturday, December 15, 2007

Erlang, Scalability - The Next Wave

I had a great time watching the TV series Surface on the SciFi Channel, with my son who is 10. In the final episode, Marine Biologist Dr. Laura Daugherty warns us that Tsunamis waves come in sets.

The first wave of the Erlang Tsunami was realizing Erlang's ability to harness the potential of multi-core CPU's to perform computations, by using concurrently executing threads and processes.

I had not recovered from the first wave when I was hit by the next wave in the set. The second wave is Erlang's ability to scale. Usually, scalability is used as characteristic of an architecture, system or technology. I don't recall if I have seen scalability used to characterize a programming language. Certainly, most programming languages provide features that can be used to develop scalable systems - but what would it take to make the language inherently scalable?

loop(State) ->
{call, From, Request} ->
{Result, State2} = handle_call(Request, State),
From ! Result,

Function loop(State) waits at the receive statement for a message that matches tuple {call, From, Request}. When a matching message is received. it is handled by the statement that calls the aptly named function, handle_call. From is messaged with the Result returned by function handle_call.

So, the received message includes information that identifies the operation to perform, the return address, and perhaps some parameters. This operation is "stateless" because the state required to process the request is provided with the request and is not maintained by loop. This is how REST works.

A key-word used in the last couple of paragraphs is "message". Note that the only coupling between sender and receiver in this message passing approach is the "shape" of the data passed (e.g. the tuple {call, From, Request}). This results in a very dynamic, and decoupled approach.

Erlang is highly scalable because its stateless, message-passing approach is suitable for all levels of distributed computing. It matters not if the distribution is among threads in one process or among hosts arrayed around the world. Erlang programmers use the same syntax, patterns and idioms, regardless of the level of distribution.

This is true scalability.

The Tsunami metaphor might have seemed like an over-statement, hopefully I have proved otherwise, but let us consider one final point. The Erlang programmer's skill scales. If a programmer can develop programs using concurrency and Erlang, it matters little if the program runs stand-alone on one box, or is widely distributed across the internet.
Do you feel the wave?


Jeff Moser said...

Nice post! It's cool to see how a language that was developed almost entirely for fault tolerance works beautifully in a multi-core world.

Let the waves hit! :)

Mike Petry said...

Thanks for suggesting that I look into Erlang! This afternoon I am playing with little executable code fragments that recursively process lists and the elegance is amazing!