Syntax in Lisp

Symbolic Expressions (or S-expressions) are the Lisp syntax for list data, as parentheses-delimited lists of space-separated S-expressions, recursively. Code is represented in S-expressions, with the first element of a list being the function to be called or special form to be evaluated.

Lisp's grammar is incredibly simple in comparison to most:

  1. A reader function takes in a stream of character objects and dispatches on those characters according to a modifiable table, which refers to further functions called reader-macros - even the parentheses in lisp syntax are implemented in this way.
  2. These functions often take in some part of the rest of the stream and collect them into objects (usually with list container structure), recursively.
  3. The resulting object is then passed to the evaluator, which can act in varying ways on that.

This enables an incredibly simple way to create domain-specific language extensions to Lisp's syntax, and since the grammar is of recursive-descent and the reader functions may act arbitrarily, grammars of sub-expressions in some domain language are not even required to be recursive-descent in their type.

Even the standard Lisp languages use these to define, say, keyword symbols which evaluate to themselves (and they also cannot be bound), so that function parameters may be passed with name instead of by their position within the function form.

Lisp's list-shaped syntax and its list-manipulation functions allow for processing even between read-time and compile-time, by macros, which are syntax-expanders that act in the full compile-time environment and language. Macro calls are recursively-expanded, until all remaining calls are core language functions. The final output is passed to the evaluator (an interpreter or compiler).