newLISP: the language
The first notable difference is related to functions. First of all, unlike Common LISP (but in a manner similar to Scheme’s), newLISP evaluate the operator part of an expression before applying it to its arguments. Furthermore, lambda expressions are not of a special data type, but actual constants which evaluate to themselves. These are subtypes of the list data type, and they behave like any other first-class obect, allowing you to manipulate them freely.
A notable aspect in this regard is the fact that (again, just like Scheme) newLISP uses the same symbol space for variables, primitives and user-defined symbols. However, unlike Scheme and Common LISP, all variables are dynamically scoped by default. Static/lexical scoping is acheived by enabling an original mechanism called contexts, which offer an arguably simpler and more transparent method of defining namespaces and building modules. Contexts are, however, more than just funky names for a namespace: contexts define lexically closed spaces as first-class objects, inside of which namespace scoping is dynamic; newLISP allows mixing lexical and dynamic scoping, allowing programmers to use contexts in order to build function factories, functions with state and software objects.
A third (and last) purely language-related detail we will discuss is that of implicit indexing. newLISP developers consider this to be a logical extension of LISP’s evaluation rules — an opinion with which it’s hard to disagree. Essentially, it boils down to replacing this:
(set 'myList '(1 1 2 3 5 8 '(13 21))) (nth 4 myList) => 5
(myList 4) => 5 (myList '(6 1)) => 21
To those who are unfamiliar with LISP and are left wondering why is array indexing such a big deal, a quick reminder: in LISP, lists (which are not the same as arrays) are a fundamental data type. Even a LISP program is a list that can be freely manipulated. Therefore, list manipulation really is a big deal in newLISP.
Implicit indexing (which is optional) does indeed look fairly Lispish, and it does seem a logical extension. However, given newLISP’s shared symbol space, its advantages can be debated. On one hand, the syntax is certainly a lot cleaner (especially in the second case, which would otherwise have to be written as (nth 1 (nth 6 myList))). On the other hand, unless the name makes it clear that you are accessing an element of a list and not calling a function, this formula can create confusion.
Two are the notable implementation features that we will discuss: the structure of lists and garbage collection. Although the differences between newLISP and other dialects are multiple, both in terms of architecture and in terms of syntax, we’ll only examine those that have greater practical importance.
In most LISP dialects, a list is implemented as a set of dotted pairs (sometimes called cons cells). Ignoring a great deal of details, a cons cell has two parts, called car and cdr for historical reasons. A list is made out of cons cells where the car (the “head” of the list) contains an atom — pedantry aside, a value –, and the “cdr” (the “rest” of the list) contains a pointer to the next dotted pair. More atoms glued together as dotted pairs form a list.
The reason of this otherwise fairly complicated scheme are mostly historical. newLISP does not have dotted pairs. Instead, every cell contains an object as car and a pointer to another object if the cell is in a list. Therefore, the behavior of cons cells is different in newLISP compared to other dialects — and extra care should be taken by those developers who like to abuse conses. The overall practical impact is fairly low though, since in most cases you do not deal directly with dotted pairs.
The part which does have a serious practical impact is garbage collection. Historically, this has been a major area of research, and (ironically enough) LISP is one of the languages that started it all. Modern Common Lisp implementations have efficient, but fairly complicated garbage collection schemes.
newLISP implements a simpler mechanism, where each object is only referenced once (ORO) — the only exceptions being symbols and contexts. This way, garbage collection becomes trivial: every object that is not referenced by a context or symbol reference can be recycled once the evaluation reaches a higher execution level. By means of a fairly simple mechanism of optimization so as not to spend too much time recycling cells, newLISP’s garbage collection system offers a faster evaluation time and a much smaller memory and disk footprint compared to other LISP interpreters.
The tradeoff for this is that, in order for the system to work, arguments have to be passed by value, and at this point you can hear C programmers leaving the room. This is only one side of the truth — since newLISP does actually allow arguments to be passed by reference, but it is not straightforward (it requires the use of contexts). Whether this is too much to endure really depends on who you’re asking and on the application you are building though. In most of the cases newLISP was built for, the overhead is really negligible, and the tradeoff is clearly worth it in terms of simplicity and size. However, some types of programs (such as scientific applications) where passing huge data structures as arguments is inherent, efficiency will require a short detour. Syntactically and practically this is not much of an issue, but it looks unclean.
As we mentioned earlier, newLISP aims to be a pragmatic, clean and expressive scripting language, though not in the semi-pejorative meaning of ‘scripting’, but rather as outlined by Ronald Loui in his In Praise of Scripting: Real Programming Projects. In order to achieve this purpose, newLISP has managed a few fairly remarkable features in terms of how it’s implemented and how it can be used.
newLISP runs on several platforms, including the well-known triad of Windows, Linux and OS X. However, it also runs on Solaris, various BSDs and Syllable. It also sports a Java-based GUI server which can use the system’s native widgets for GUI, but it also offers interfaces to GTK, Tcl/Tk and OpenGL for those cases when speed or portability are at stake. Indeed, the GUI server feels somewhat slow — partly due to its architecture, and possibly in part due to, well, Java. The library’s GUI abilities are also fairly limited (although more than enough for what it is meant to do), and there is no GUI builder available. Then again, we can freely argue that newLISP wasn’t really built with this in mind, and the chores of manually setting up the layout aren’t all that tragical when your application needs only a handful of widgets.
Apart from this, newLISP also comes with a handful of extra modules, touching a variety of subjects, from regular expressions and Amazon-AWS EC2 to network sockets and database interfacing. It also has an import function, which allows interfacing with DLLs and shared libraries.
Although all these standard modules are fairly limited in terms of capabilities, they do have the merit of being reliable, simple to use and understand and very well documented. As a consequence, building up further functions as required can be done with reasonable ease. The only current setback is that third-party modules are somewhat behind the changes newLISP has undergone in its transition to the recently-launched 10.0 version. As a consequence, their behavior may include unexpected failures and green dwarfs, as justly signaled on the newLISP homepage.
A very interesting feature of newLISP is its clean and very easy model of distributed evaluation. newLISP can run as a TCP/IP daemon, and through the use of a single function, net-eval, remote newLISP servers can be given instructions to execute. While high performance computing fans are already thinking about how you can parallelize LISP programs more efficiently, those who frequently write small scripts will appreciate the extra flexibility given to them. The implementation may not be particularly efficient in terms of performance, but it is efficient in terms of ease of use and flexibility.
newLISP also takes reasonable steps towards making it easy to distribute applications, which has traditionally been a chore in the LISP world, only comparable to that of propperly managing memory in C. The newLISP executable is small and it has a very tiny memory footprint. It can also be compiled as a UNIX shared library or as a Win32 DLL, so it can be embedded in other programs. The other option of interaction is the use of I/O pipes and network ports. The former is used by BBEdit, and the latter is used by newLISP’s Tcl/Tk frontend, among others.