Zen for Embedded Systems

Zen is a tiny Forth with a big purpose: It is a model Forth designed for readability. Have you ever noticed that there are no intermediate Forth programmers? You are either a rank beginner or a black-belt guru. Why? My guess is that after reading Leo Brodie's Starting Forth (or my own Mastering Forth), you are told that the next step to learning Forth is to read the source code to the language itself. This sounds like a good idea; unfortunately, the source code of a professional Forth system is likely to be tightly packed, handcoded, over-optimized, and generally incomprehensible. If you can make it through all of that, you are in a good position to write a book of your own.

What you need is a Forth that is written mostly in ... you guessed it! Dr. Dobb's Journal has a long tradition of publishing personal implementations of computer languages, from Tiny Basic (January 1976 issue) to Small C (May 1980 issue). So to continue the tradition, we are proud to present ZEN, the tiny Forth.

Programs written in ZEN can be quite small, about half the size of programs written in C. In these days of megabyte memory, is small size important? Yes. Small size often means high speed. On-chip memory generally runs at least twice the speed of off-chip memory. A 68HC11 has only 8K of internal masked ROM. The most popular DSP chip, the TMS32010, has only 4K. Even if the density of on-chip memory doubles each year, three years from now microprocessors will support only 64K of internal ROM.

In recognition of the importance of small programs in embedded applications, ZEN is inherently ROMable and easily fits in an 8K PROM. Not only that, but ZEN is extensible in ROM. How can a program grow in ROM? Assume that ZEN is resident in a single-board computer (SBC) and is talking to a host computer over an RS-232 serial line. Put a RAM chip in an empty ROM socket, and then send source code to ZEN with instructions to compile it to the address occupied by the RAM. ZEN will write the program there but will assume that all references to RAM point to a separate address space, normally the on-chip RAM.

Test the program interactively and, once it works, tell ZEN to read the RAM image and burn it into a PROM. Finally, remove the RAM chip, plug in the PROM, and you're done. The idea of running the development system in the target hardware may seem novel; what it means is that all testing and debugging take place in the target system. The host computer is used only as a terminal and disk server. The chances of a working program not working when you burn the final PROM are utterly remote.

The ANSI Forth standardization effort is well under way. The current document, ANS X3J14 BASIS 9, is available for $6 from the X3J14 Secretariat (c/o FORTH, Inc., 111 N. Sepulveda Blvd., Manhattan Beach, CA 90266). The BASIS is modified at each meeting until it becomes the draft proposal that, after a period of public review and revision, becomes the ANSI Forth Standard. ZEN 1.7 is an unofficial implementation of BASIS 7 and provides all required words. It supports the double number, file access, and BLOCK extensions.

The stack manipulation words found in ZEN are:

  DUP DROP SWAP OVER
  ROT ?DUP
  2DUP 2DROP 2SWAP 2OVER 2ROT
  >R R@ R> 2>R 2R>
  NIP TUCK PICK ROLL

ROLL is defined in high-level because it cannot be efficiently mapped into machine code. You will find that 2>R and 2R> obviate almost any need for ROLL. In case you are not familiar with NIP and TUCK, they are equivalent to the phrases SWAP DROP and SWAP OVER, respectively. NIP is especially handy and is a single instruction on the Harris RTX 2000 Forth Processor. It only takes a single instruction (plus NEXT) on the 80x86 as well.

The single-precision arithmetic operators form this set:

   + - * / MOD /MOD */
   1+ 1/ 2* 2/
   NEGATE ABS  MAX MIN

1+ and 1- increment and decrement, respectively. 2* and 2/ shift once right and left, arithmetically. Notice that 2+ is not available. Use CELL+ instead to move to the next cell in an array. Use CELLS instead of 2* to change cells into bytes, as in:

  1024 CELLS ALLOT.

Many single-precision operators have double-number equivalents.

   D+ D- D2* D2/
   DNEGATE DABS DMAX DMIN

Multiplication and division generally mix precisions to avoid calculations with unnecessary accuracy.

  UM* UM/MOD  */MOD M* M/MOD
  M+ S>D D>S

A full complement of logical and comparison operators exists.

   AND OR XOR NOT
   0< 0= 0> < = > U< WITHIN
   D0= D< D= TRUE 0 1

WITHIN is especially powerful because it compares within in a circular number space. It can be used to control the duration of an event by comparing the current value of a self-incrementing counter to some future value. All other comparison operators can be based on WITHIN.

These are the memory access primitives:

  @ ! C@ C! 2@ 2! D@ D!
  CMOVE CMOVE> MOVE +!

D@ and D! are equivalent to 2@ and 2! except that the order of cell storage is not specified. MOVE is a smart CMOVE that moves without overlapping bytes. It cannot be used to fill memory with a character.

Here are the flow-of-control operators:

BEGIN WHILE REPEAT UNTIL AGAIN
IF ELSE THEN DO LOOP +LOOP
IJ LEAVE UNDO EXIT
EXECUTE @EXECUTE

UNDO removes one level of DO ... LOOP support from the return stack, allowing EXIT to execute within the loop. UNDO EXIT is the preferred method for leaving a word from inside a loop. @EXECUTE is exactly equivalent to @ EXECUTE, but is much faster.

These are the number conversion and formatting commands:

  BASE DECIMAL HEX
  CONVERT VAL?  PAD
  <# # #S SIGN HOLD #>
  . U. D. D. R

Output conversion and formatting takes place at PAD 1 - and downwards, leaving PAD itself free for applications. VAL? converts strings to numbers, and BASIS 6 defines numeric, string, and character literals.

[ASCII] ASCII "ccc" LITERAL

ZEN adds DLITERAL, the double-number equivalent of LITERAL. ASCII and [ASCII] make the following character into an ASCII literal inside and outside a definition, respectively. The string literal operator "(quote) works only inside a definition and accepts the following string of characters, up to the next", as a string literal.

   COUNT SKIP SCAN /STRING
   FILL BLANK ERASE -TRAILING
   STRING." EVALUATE

SKIP SCAN and /STRING are natural factors of WORD. SKIP and SCAN closely map the 80x86 string operator SCAS. /STRING is a generic substring operator; it can be used to select any part of a string. STRING is the fundamental string compiler and is used by the "and." commands.

EVALUATE is the most powerful string operator: It interprets any string. EVALUATE is used for forward referencing, macro definition, and any other instance of late binding. For example, coldstart uses the phrase READY EVALUATE for device initialization. Suppose you add a device that requires its own initialization:

:READY ( new initialization) READY;

READY is redefined to include the new initialization. READY EVALUATE executes this new version, which in turn executes the previous version.

The interpreter level uses these words:

   BLK >IN     TIB #TIB     SPAN     STATE
   FORTH     CONTEXT     '    FIND     WORD
   CR TYPE      EXPECT KEYS KEY?
   KEY EMIT PAGE MARK TAB
   BL   SPACE SPACES    .((    DEPTH

Most Forths have words such as KEY?, which is true if a key is available; PAGE, which clears a screen or page; MARK, which TYPEs in emphasized mode; and TAB, which repositions the cursor to a given X Y coordinate. None of these words have been standardized. In addition, ZEN supports KEYS, which accepts characters without echoing or editing. In general, if the input stream is coming from a human being, use EXPECT; otherwise, use KEYS.

The compiler layer adds these words:

   CURRENT DEFINITIONS [] ['], C,
   COMPILE [COMPILE] IMMEDIATE
   RECURSE

RECURSE allows a definition to reference itself. Remember, though, that the size of the return stack is likely to be limited. The following defining words are provided:

CREATE VARIABLE CONSTANT :;
2VARIABLE 2CONSTANT
VOCABULARY

Two other defining words are used to build ZEN: USER and XFER. USER creates a user variable, whose address is based on its offset from the address in the pseudoregister u. A user variable is private to each task in a multitasking environment. XFER creates a transfer command, whose action is specified by its offset in an execution vector pointed to by the user variable x. By changing this execution vector, input and output can be redirected to different devices or files. Because x is a user variable, each device can be controlled by a different task, so a background task can spool text to the printer while a foreground task paints graphics on a screen.

The RAM and ROM address spaces in ZEN are kept entirely separate. The dictionary, machine code, and tables are built in ROM, and the data fields of variables and arrays are built in RAM. Each word that refers explicitly to RAM has an equivalent that refers instead to ROM.

RAM                      ROM
----------------------   -----------------------
CREATE                   VARIABLE
C, ,ALLOT
HERETHERE
DOES>                    GOES>
>BODY                    >DATA

Let's say you want to make a table of powers of ten:

  CREATE TENS   1 , 10 , 100 , 1000 , 10000 ,

The TENS table will be built in ROM. Suppose you want an array of seven cells:

  VARIABLE DAYS 6 CELLS ALLOT

The DAYS array will be built in RAM. Notice that VARIABLE allocates the first cell. It is not possible to initialize DAYS because anything stored in RAM disappears when the power is turned off. You can use GAP to allocate uninitialized cells in ROM, but you will rarely need to do so.

DOES> refers to the body of a defined word in ROM.

:COLOR \ set CREG to the value n.
    CREATE ( n), DOES> @ CREG!;

8 COLOR RED 12 COLOR GREEN

Executing RED will store 8 into CREG. DOES>DATA to the data field of a defined word in RAM.

:KOUNTER \ self-incrementing object.
VARIABLE DOES> DATA 1 SWAP +!;

KOUNTER ALPHA KOUNTER BETA

Each execution of ALPHA will increment its value. To read this value, you must be able to recover the data field address.

  ' ALPHA >DATA ? (prints value)

Use >BODY to recover the parameter field address of an object in

  ROM: ' RED >BODY ? (prints 8)

The restart and error-handling mechanism, designed by Wil Baden, was published in FORML 1987. The Forth command-line interpreter is QUIT; it is the default Forth application.

  :QUIT RESET
     BEGIN CR QUERY interpret OK?
  AGAIN;

When Forth begins executing, it goes to ABORT and from there to QUIT. The function of ABORT will be performed by every error recovery sequence. ABORT is the default Forth error restart sequence.

  :ABORT
  BEGIN PRESET QUIT GRIPE AGAIN;

In QUIT, the work of clearing the data stack is factored into RESET. The data stack, however, is not completely cleared; one item is left, but this has no impact on existing programs. In ABORT, PRESET clears both stacks all the way to the bottom.

You now have a way to customize the error recovery sequence. In addition, ABORT shows the pattern for dedicating an application: The default application is QUIT, and the default error recovery sequence is GRIPE. GRIPE sounds like TYPE and takes the same parameters, address, and length. It displays the error message provided by ABORT. It may also attempt to display the name of the word being interpreted.

PRESET leaves both stacks empty. Invoking the next word will put the address of the return point on the return stack. RESET leaves that address alone. A word can get back to that location by RESET EXIT. On the return, the parameters for an error message string are assumed.

The default application and default error recovery sequence can be changed, and ABORT" will still work with them.

  :err RESET;

  :ABORT" [COMPILE] IF [COMPILE]"
  COMPILE err [COMPILE] THEN;
  IMMEDIATE

err in the definition of ABORT" cannot be replaced with RESET, but you can build a variable error message string and follow it with RESET EXIT or RESET; to emulate ABORT".

The following trivial example changes the default application to indent three spaces for every value on the stack before receiving new input. This process is preventative debugging: If you are surprised by the indentation then you are in trouble. Also, the example demonstrates logical structure as you compile a definition from the keyboard.

The error recovery message is changed as well.

:OLD-ABORT ABORT;

:INDENT DEPTH STATE @ -3*SPACES;

:SHELL
     BEGIN CR INDENT
           QUERY interpret OK?
     AGAIN;

:ABORT
    BEGIN PRESET SHELL
          CR." Abort:" GRIPE
    AGAIN;

ABORT installs a new application and error handler. OLD-ABORT reinstalls old application and error handler.

With PRESET and RESET you can establish intricate error recovery and restart networks. The following generic QUIT can be used to restart the default application in most implementations. The idea is to return to the position just before you invoke your default application.

  :QUIT RESET R> CELL - >R;

ZEN supports both text files and traditional Forth BLOCKs. The text file support includes these words:

CREATE-FILE   OPEN-FILE              CLOSE-FILE
DELETE-FILE   RENAME-FILE
READ-FILE                            WRITE-FILE
SEEK-FILE                            FILESIZE                 FILEPOS
READ-LINE                            WRITE-CR
IO-RESULT

READ-FILE and WRITE-FILE read a sequence of bytes from the current position in the file to and from a buffer in memory. SEEK-FILE and FILEPOS change the position of a file, and FILE-SIZE returns its size. READ-LINE reads a line of text, and WRITE-CR writes an end-of-line terminator. To write a line of text, use WRITE-FILE followed by WRITE-CR. All file primitives work with a file identifier returned by CREATE-FILE or OPEN-FILE, so multiple files are supported. Error codes, if any, are returned in IO-RESULT.

The BLOCK support includes these words:

  BLOCK BUFFER UPDATE
  SAVE-BUFFERS FLUSH LOAD

You can find the source code and executable image of ZEN in the file ZEN190.ARC on CompuServe in the DDJ Forum. ZEN is also available on the GENIE, ECFB, and BIX bulletin boards. Execute the file ZEN.COM and press <RETURN>. You should see the OK message. ZEN has a small set of debugging words you can use to look around:

  WORDS .S ? DUMP

You can compile and test definitions from the keyboard, or type GO to compile the text file KERNEL.SRC. Use any text editor to edit this file. I use the Sidekick pop-up editor.

Use GUARD to protect all words in the dictionary from FORGET. EMPTY restores the dictionary to the previous GUARDed word set.

I hope you enjoy ZEN and find it useful. Please send all comments and criticisms to me.