Programming Erlang

gnalre writes "Every day it seems there is a new publication of a book on perl/python/ruby. Some languages however do not seem to get that sort of attention. One of those under-represented languages is Erlang, however for the first time in 10 years a new Erlang book has been published. As someone who had a brief flirtation with Erlang long ago, I was interested to see how the language had evolved in the intervening decade. I was also curious to re-evaluate Erlang to see what solutions it offered to the present day issues of writing reliable distributed applications." Read on for the rest of Tony's review.

Programming Erlang - Software For A Concurrent World
author	Joe Armstrong
pages	515
publisher	The Pragmatic Programmers
rating	8/10
reviewer	Tony Pedley
ISBN	1-9343560-0-X
summary	Parallel programming the easy way

Programming Erlang — Software For A Concurrent World (ISBN 10193435600X) is part of the pragmatic programmer series. As with all the books in this series, it is available in paperback or for a reduced cost you can directly download it in PDF format (which is always useful if you spend a lot of time on the move and you do not like carrying around a dead tree with you). The book's format and layout as with all the books of this series are clear and logical.

The book is written by Joe Armstrong, who co-authored the first Erlang book a decade ago. He was also one of the originators of the Erlang language and has been directly connected to its development ever since. We can therefore be assured about the author's knowledge and insight into the language, if not his impartiality.

The book itself can be roughly split into three main sections: Getting started and Sequential programming, Concurrent Programming and Erlang libraries and advanced Erlang techniques.

In Chapter 1 the author sets out his stall of why Erlang is worthy of your attention. It's clear from this chapter that the author feels Erlang's strength lies in applications requiring an element concurrency and fault tolerance. Another emphasis is made of running Erlang on modern multi-core processors, something that was only a glint in a hardware designer's eye 10 years ago, but is rapidly becoming an issue in all areas of programming. From this chapter you also get a feel on how the author approaches his programming in that he states that he wants the reader to have fun with the language, which is a refreshing change to some language text books whose main purpose appears to be as a cure for insomnia.

Chapter 2 goes through installing Erlang and the Erlang shell (a command line environment similar to ones with languages such as perl). The chapter also starts us into the strange world of functional programming, where variables can only be given a value once (e.g you cannot do i=i+1), recursion replace loops and pattern matching replaces assignments. Fortunately the Erlang language is remarkably concise. For example there are only 4 data types. However to those coming from a purely procedural programming background the learning curve could be a steep one. Saying that the Author does a good job of leading you through the languages intricacies with examples being compared to code from languages such as Java to help keep your feet on solid programming ground.

The next 3 chapters move on to writing simple Erlang programs. As a quick aside, for anyone new to Erlang it is well worth examining the quicksort implementation described in chapter 3. Its conciseness and simplicity was one of the reasons the language won me over when I first met the language.

These chapters also cover error detection and handling. It's worth noting that Erlang has a philosophy of ensuring programs fail hard, so that bugs can be weeded out at an early stage. This idea very much defines how Erlang error handling is defined.

One criticism of the first section is Chapter 6, which describes compiling and running an Erlang program. I would have preferred that this information be covered earlier in the book or be placed in an appendix because it is probably an area you will want to reference repeatedly.

Chapter 7 is where things really get interesting and the true power of Erlang starts to come to the fore. This is where Erlang's concurrency credentials are explained. This chapter begins by providing some useful metaphors of the Erlang concurrent model, but chapter 8 is where the fun begins by describing the Erlang concurrency primitives that allow the creation of processes and the process communication methods. The author here highlights one of the language features, the Erlang light weight process. These are true processes (not threads) but take up very little in the way of resources. Indeed it is not unusual to have 1000's of such processes running in an application.

The next few chapters expand on the available concurrency primitives and how to move from concurrency on your local processor to concurrency utilizing the resources of multiple machines either on a local network or across the web. It finishes the section off by showing the example of a simple IRC application.

Chapter 12 starts the next section by looking at how to interact with the world outside the Erlang environment. First it examines how to interface an Erlang program to applications written in other languages such as C. It then goes onto to look at file and socket handling in Erlang. Chapter 15 looks at two important Erlang storage primitives ETS and DETS before we get to the OTP Erlang libraries in Chapter 16.

The OTP libraries are the standard Erlang libraries and tools. In fact the OTP libraries are worthy of a book in itself. The author highlights the section on the generic Server module as the most important section in the whole book and one to be reread until its importance has sunk in. This is because here are encapsulated many of the lessons learned in writing industrial fault-tolerant applications, such the updating of a running applications code without causing that application to miss a beat. The section is finished off by describing the Erlang distributed database (humorously named Mnesia) and then finishing it off with the example of a simple server application.

The book finishes off by looking at Erlang on multicore systems including its support for SMP. As the author states this is the leading edge of present day Erlang and is still under development.

I would like to thank the pragmatic programmers for publishing this book. Erlang's profile has been in need of highlighting for many years and hopefully this book will help. The book definitely provides a great starting point for anyone who wants to get to grips with the language and takes them to the point where they can start writing useful applications. This book is a worthy successor to the last book published and does a good job of both updating the material and explaining some of the later developments such as SMP. Anyone who has a need for writing fault tolerant applications should at least look at this book. If nothing else you will never be afraid of dealing with recursion ever again.

In many ways the book cuts off just when things are getting interesting. There are hints in the book about real world Erlang's applications and it would have been good if some of these experiences could have been expanded. Hopefully this book is the start of increased exposure for Erlang. If so then someone may get around to writing another Erlang book describing some of the advanced issues about generating robust applications. I just hope it won't take another 10 years this time.

Tony Pedley is a senior engineer specializing in real-time embedded systems. In his spare time he likes to tease windows programmers and confuse managers by telling them it would be a lot easier if we wrote it in Erlang.

---

You can purchase Programming Erlang - Software For A Concurrent World from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Try it out

[romiz]

The first chapter [pragmaticprogrammer.com] is avalable online to get a taste of the book (and the language).

Great book creation process

[DoofusOfDeath]

I bought the book while it was still being written. I was able to download drafts, and (if I had the time) submit bug reports. When it was finally done, I got a printed copy in the mail.

I haven't had much time to play with Erlang (or the book) yet, but it was a really nice feeling to be able to get early access as long as I was willing to see unpolished content. Bravo, publisher.

apropos erlang (Go Sweden!)

[Jhan]

The Ericsson Language, for those who have not been exposed to it, is a programming language meant to make concurrent (multi-process, multi-processor, multi-machine) programming really simple. It's used on many AXE telephone base stations world-wide. It has great message-passing support, and a pretty nice library. It is actually not a functional language, but a logical one. Basically a perverted version of Prolog.

Wings3D

[UglyMike]

Strange that I didn't see Wings3D mentioned yet. ( http://www.wings3d.com/ [wings3d.com] )
It's an open-source subdivision surface modeler held to great esteem in the modeling scene

It is also an Erlang application....

Review worth a +1 karma!

[Z00L00K]

The review is well written, and even though Erlang is a programming language that not everyone is used to program in it's well worth to mention that a large number of telecommunication devices from Ericsson are running software written in Erlang, so most people has probably been using it without knowing it.

And in my opinion; If you are familiar with more common languages like C and Java you should take a deeper look into Erlang unless you prefer to study Prolog or Cobol. Just take a dip or a deep plunge, you never know when you end up in a project where knowing Erlang may prove useful - it is actually developed to be used in real applications and not as a theoretical study object.

And Erlang is designed to handle concurrent programming from the bottom, which is a real problem in large multi-user systems. You can of course use C or Java and solve concurrency problems with semaphores or synchronization, but the solution in Erlang may be much more elegant.

And for all of you that are familiar with the Eclipse development environment; There is a plugin called Erlide [sourceforge.net].

Re:The Seven Deadly Sins of Erlang

[ibullard]

That is the same guy who says you can get artificial intelligence from the bible [rebelscience.org]. He's a loon with a large vocabulary.

Re:Name mixup?

[mooingyak]

It was actually named after the same guy.

http://en.wikipedia.org/wiki/Erlang_programming_la nguage [wikipedia.org]

Re:What's missing from Erlang...

[be-fan]

But that's what I'm holding out for, is a high-level language with good performance that at least approaches C.

There's basically a handful of languages that would meet your needs here. On the dynamic side, you've got Common Lisp. On the static side, you've got ML, O'Caml, or Haskell.

After that, your options trail-off significantly. The other dynamic languages are all much slower than Lisp (more in the league of Python than in the league of C), and the other static languages (C#, Java), are much lower-level/less productive.

Full-time Erlang programmer gives his view :]

[pnis]

As somebody writing code in Erlang as a profession, I say, though it has strength, it's crap. It has strength in the efficient handling of many processes (faster then the OS when there are many many thousands of them), aand (re)loading modules on the fly into the virtualmachine is nice. But on the flipside Erlang is slow as it's a very high level language, with a not so clever virtual machine - much less so than java vms (well I have to admit I am an assembly&C fan). Also functional programming my ass. In reality it's more like very basic barebone declarative programming, where you can only use global variables (mnesia database, and in worse cases the process dictionary) and there are no loop constructs (you can only use ugly additional functions and recursion instead of a "for" construct). Well, at least this is my opinion.

Re:I know why it's been 10 years

[be-fan]

So, think, what are you actually trying to accomplish in the first block of code, in terms of the end-result.

You're computing a list, target, where each item is computed by F(G(x)), where x is an item from source_list, taken in order.

Note that stuff like "iterating over source_list", and "appending to target" are not really important. They are certain necessary steps in one possible implementation of what you're trying to accomplish, but they're just incidental --- they're not part of the end result.

Now, map is a function that takes two arguments: a function that takes one parameter and a list. It constructs a new list equal in length to the source list, with each element obtained by calling the function on one element of the source list. Ie: if you do map(square_number, [1, 2, 3]) you get [1 4 9].

Now, let's look at the functional code. The first line is a function definition for the function compose2.

compose2 = lambda F, G: lambda x: (F(G(x))

lambda is a way of introducing a literal function, much like "1" is a literal integer. lambda F, G: ... means a function that takes two parameters, F and G, and returns ... (note the return is implicit. In this case, the compose2 variable is assigned to a lambda that takes two functions F and G, and returns a third function that takes a value x and computes F(G(x)). Note that this is exactly the function we want to use to compute our list.

Now let's look at the second line:

target = map(compose2(F,G), source_list)

The code first calls compose2 with parameters F and G to get a function that computes F(G(x)). Note that this function is exactly in the form expected by map. Then it calls map with that function, and the source list. Map will internally apply F(G(x)) to each x in source_list, and retrn a list containing the results.

Note that usually stuff like compose2 will be a standard library function, because this sort of thing is so common. In that case, your original five lines of code, becomes a single line, that expresses exactly the end result you want, without being cluttered with incidental details like iterating over or appending to a list.

I should point out that I don't know Erlang, or whatever language that snippet is in. The second example is nonetheless easy for someone used to functional programs to understand, much like the first example is easy for an imperative programmer to understand, even one that doesn't know Python.

Re:I know why it's been 10 years

[Just Some Guy]

At this point, I don't even understand what situations this could be usefully applied to.

Well, functional programs have a lot of other restrictions, but it boils down to this: with very rare exceptions (such as file or socket handling), functions don't have side effects. That is, they don't modify a program's global state, and they don't depend on global state. You can call foo(args) 100 times in a row and each time the output will be identical. My C++ is rusty, but imagine something like:

int foo(int &bar, int &baz) { return bar++ * baz++; }

Any code that calls foo() will find its internal state changed - both bar and baz have been altered. In comparison, this is almost impossible to do accidentally in functional languages.

So, great, what does that buy you? Well, the map() function in you example knows that it's supposed to call compose2() on each item in source_list. Because it's impossible for compose2() to have strange global side effects and it doesn't depend on external state, map() is free to fork() 10 times and let each child process munch through independent subsets of source_list. In the case of Erlang, map() is also free to hand off chunks of source_list to other machines and coalesce the results from all of those hosts at the end. As long as compose2() gets called on each item at some point, the order (and indeed the physical location) of each call doesn't matter a lick.

Imagine that you have 20 quad-core systems in a cluster. Replacing the imperative example above with the functional example gives your code an 80x performance boost, minus a little network overhead. Does that sound more interesting now?

Have you ever written a loop?

[Estanislao Martínez]

It's very different, but the big advantage is that it's higher level than the stuff you confess to understanding better.

The code in question (in Python? not a great choice for doing an example!) uses two very common higher-order operations in functional programming: map and compose. A map operation takes a complex data structure (most common example: a list), and a function that applies to elements of that data structure, and returns another structure, with the same "shape," where each element in the result is related to its corresponding element in the original structure by being the result of apply the function. Thus, if you have a list [2, 3, 5, 7], and a function inc that increments a number by one, map(inc, [2, 3, 5, 7]) evaluates to [3, 4, 6, 8].

In the case of a list, map is can be implemented by creating a new list of the same length as the original, looping over the list, applying the function to each value, and storing the result in the result list. This is a kind of task that imperative programmers find themselves doing all the time. The problem with this, however, is that if you're writing code like this all the time, you're writing at a much too low level, with the all the disadvantages of that:

1. It takes you longer to write code. Instead of writing just result = map(fn, myList), you have to write a loop.
2. Your code is specifying the mechanism to convert the input list into the output list over and over, instead of describing the relation between the lists. Suppose you want to change the mechanism for doing this sort of operation, e.g., to parallelize it. In functional style, you can just rewrite map to do its work in parallel. In imperative programming, you have to work harder.
3. The concept of mapping a function over a structure, to produce a new structure with values related to the original by the function, applies to many kinds of data structure, not just sequences. You can write equivalents of maps for any container data function you like, even though the mechanics are different; e.g., to implement a map function over a tree you typically use recursion. This means that in the imperative style, the code for "increment all the values of this list by 1" looks very different from the code for "increment all values in this tree by 1"; in functional programming, they will at least look very similar, if not completely the same.
4. There are many, many other common progamming blocks that you can abstract into a function in functional programming, but not in imperative programming. For example, filtering a list to only keep the elements that satisfy a boolean function, which in functional programming is usually just a function called something like filter. For example, if you have a function called even, that returns true only for even numbers, here's how you get the even numbers in myList: filter(even, myList)
5. Functional programming solutions often compose more easily than imperative ones; i.e., it is easier to put them together to solve more complex tasks. For example, if you need to increment a list of numbers by one each, and only keep the ones that are even in the result, you can do that easily and transparently: filter(even, map(inc, myList)). In the imperative style, the patterns for mapping and for filtering don't combine this straightforwardly.
6. Functional code is usually easier to read. When I read a piece of imperative code and come across a loop, I normally have to work the steps a bit in my mind in order to see what it is that it does: "Hmm, we create a new list at the beginning, then we loop through the elements of this other list, and as we go through each one, we apply a function to it, and put the result in the list we first created. Ah, we're mapping over the list." In functional programming, these common patterns find immediate expression in a higher-order function

Re:Be careful

[be-fan]

Nobody live-updates a running Erlang (or Lisp, which does the same thing), without deploying the code into a test environment first. The thing that live-update let's you do is take a well-tested and well-debugged piece of code and inject it into a running service without stopping the service.

AT&T uses Lisp for some of their call routing equipment for the same reason. If you need to add a new feature to a switch, you cannot drop the call's it's handling to swap out the software.

There are also lot's of uses of live-updating in non-productive environments. For example, it can be used as a development model, drastically reducing the edit/compile/debug cycle. When I write Lisp code (I presume Erlang programmers would be similar here), I rarely restart the program I'm debugging to recompile it. For example, my current Lisp image has an uptime of about 2 days. That means for the last two days, I've been incrementally compiling and testing my changes into the same running process.

It's also very helpful when the test scenario for the software is complicated. I maintain some software that does data collection/analysis of spectrum traces. Testing the software involves hooking up an antenna to a big awkward machine and going outside. The program is unfortunately written in C, which means I waste a lot of time shuttling back and forth between the test environment and the compiler. If I could stop the trace in the middle, make small changes, and keep going, my testing time would go way down.

classic papers

[rsilverman]

Here is a classic paper on the style and advantages of functional programming:

http://www.math.chalmers.se/~rjmh/Papers/whyfp.htm l [chalmers.se]

Also, John Backus' Turing Aware lecture, "Can Programming Be Liberated from the Von Neumann Style?"

http://www.stanford.edu/class/cs242/readings/backu s.pdf [stanford.edu]

Re:apropos erlang (Go Sweden!)

[TheRaven64]

add(X, 0, Sum) :- Sum = X.
add(0, Y, Sum) :- Sum = Y.
add(X, Y, Sum) :- dec(X, 1, X2), inc(Y, 1, Y2), add(X2, Y2, Sum).

This is quite bad Prolog. Try this instead:

add(0,X,X).
add(s(X),Y,s(Sum)) :- add(X, Y, Sum).

Where s/1 is a successor operator, which may or may not be defined, depending on how you wish to represent numbers. Your second case is not required, although it can be used for a tiny speed boost. If you want to take this to its logical conclusion, then you can assert/1 each result that you've already unified with true. This technique is more useful in something like a Fibonacci sequence, however.

Re:That's a joke, right?

[TheRaven64]

Erlang is actually the closest language I've seen to the original meaning of Object Oriented Programming; Alan Kay's simple computers communicating via message passing. The syntax varies from absolutely gorgeous (anything involving binary data) to just plain painful (metaprogramming, string manipulation, ADTs, etc). The foreign interface layer is horrendously badly documented, and has enough overhead that it's hard to use it for the one thing where it would actually be really useful: serial code that needs to be fast.

Back to the point, you can do most of the OOP things you might want to do if you just think 'object equals process'. Erlang is really nice for some things (I had a project that wanted to run on a 64 CPU machine, and Erlang was great for that), but it could be a lot better. R11 improved things a lot over R10 though, so I'd definitely recommend learning it. Even if you don't use it much, I think knowledge of Erlang improves your ability to program in other languages, just as Lisp and Smalltalk do.

Re:I know why it's been 10 years

[Algan]

Actually FP code is an order of magnitude easier to debug/maintain/understand, once you get past the weird syntax and restrictions. The absence of side effects makes it easier to isolate a problem or to build up an understanding of what the code does. The human brain can focus on only so many things at once. Since you don't have side effects, a function's result depends only on its parameters, so once you understand it you can think of it as a black box and move up to the call chain. When you debug, you can move up and down the stack until you figure out what function returns a wrong result and then you know that the problem is isolated to that function (there are no global variables/outside environment that can affect that function)

Re:What's missing from Erlang...

[DaveTerrell]

You can do reload natively compiled modules. In fact, you can replace a native compiled module with a byte-code one and then vice versa, in a running vm. I just did it 5 seconds ago.

You can use periods and @s (and anything else) to namespace atoms, if you want. The module loader will even track down module foo.bar to foo/bar.beam... compiler support for it isn't great but it works. nobody bothers to use it though.

The syntax takes a while to get used to, but it becomes very easy to write.

What I've found is that the longer I write it, more often I crank out code for hours at a time that just works.

Re:What's missing from Erlang...

[pivo]

Yes, I understand that. But since atoms in Erlang have no associated value, I don't see how the fact that another module uses the same atom could cause any confusion. Maybe if you gave an example.

I'm considering Erlang for a project, so I honestly would appreciate knowing how this might be problematic.

Re:apropos erlang (Go Sweden!)

[Hal_Porter]

Ericsson is a Swedish company, founded by Lars Magnus Ericsson [wikipedia.org]. Erlang was developed by Swedes at Ericsson but is named after Erlang who was unfortunately Danish. It is possible some Danes sneaked in and did this naming before the security guards could throw them out.

Re:What's missing from Erlang...

[Luna-tic]

The implementation issue was that you had to choose between performance and being able to reload functions later. I would very much like it to be able to JIT or even compile down to binary (x86_64 too, pretty please?), then be able to just leave it running, and have it reload functions as needed.

There is HiPE, which compiles Erlang to native code. And you can HiPE compile modules together with normal BEAM (interpreted code), with function granularity. And then reload the modules using BEAM and/or HiPE, again and again.

But the reason for commenting your comment was the x86_64 part. HiPE has had x86_64 support for three years now. And its creation made me learn to write the x86_64 machine language in hex...

Note: I did the x86_64 backend as part of my Master's thesis.
One relevant paper: http://www.update.uu.se/~luna/papers/ppdp05.pdf [update.uu.se]