I mostly seem to post here when I am thinking about ideas which may have something to contribute to some project or other, but less often when I am just plodding along making progress. Today, though, I have spent my time actually working on the code for ELIUS and making some fairly significant internal changes.

When I first started to write it I was not sure how I was intending to lay-out the memory, so I created separate arrays for each of the major parts (dictionary, data and return stacks, string pool, input buffer and so on) Once the code for manipulating each bit individually seemed to be working, I then needed to combine them, so that (for example) dictionary entries could refer to strings in the pool. This was a bit tricky, because some of the arrays I had created were “word” (32 bit) sized, and other were “byte” (8 bit) sized. To combine them in a single byte-addressable memory area I had to make sure that all memory access went through functions (read and write byte, read and write word) which did the right things with the bytes in the array. I continued with this process of combining until I ended up with a single array of bytes (initially sized at a relatively conservative 64K). I had reached this point a few days ago.

This was a good basis for working with each of the areas of memory and their combinations, and enabled me to get the wrinkles out of the interaction between them. The big remaining problem was the way the entries in this array were addressed. Addresses in the ELIUS code were actually just indexes into this big array – the input buffer occupied bytes 0-1024, for example. This is obviously not the same as the addresses which the code will be able to use on the bare-metal Raspberry Pi, or even when running as a “guest” under Linux. This way of working also has its problems with “primitives”. In this context a “primitive” is an operation which appears like a user-defined one, but is actually implemented directly on the machine. Examples typically include basic stack, memory and arithmetic operations.

With virtual addresses defined as simple offsets into an array of bytes it’s very hard for code “inside” the virtual system to refer to these primitives, which live outside the array. So I was a bit stuck, unable to progress with implementing the key primitives I need to enable other words to be defined. Today, I took the plunge and converted the whole system to use real memory addresses. For practical reasons this also included moving all the system variables (things like stack pointers, next free dictionary slot address and so on) into the managed memory area.

Even though this system is aimed at the Raspberry Pi as a target platform, I am developing it on a wide variety of machines – that’s the advantage of using a cloud-hosted source repository (in this case a repository at GitHub) and a distributed version control system. I can hack away at the code wherever I am and whatever machine I am using. Even machines without an internet connection can still use version control, just synchronize with the main repository later.

My laptop runs a 64-bit operating system, and even though I usually develop in a 32-bit Linux VM, on this machine I have installed a 64-bit VM (for complicated reasons involving compatibility with some client work I am doing). The 64-bit platform has 64-bit pointers, so even after converting the code to use real memory addresses I still could not store the machine addresses of the primitive functions in 32-bit words. So the code (on my laptop at least) now includes some compile-time jiggery-pokery to select either a 32-bit or 64-bit “word” size.

The end result of all this is that I now have a much more useful code base. All the old tests for stack, dictionary and pool behaviour still work, but now it uses a block of “real” memory. The next step is to decide on and implement a minimal set of primitives to support the definition of further words. I’m hoping that fairly soon after that I might be able to get a deployment on the Raspberry Pi that does something other than run its own tests. Maybe flashing an LED?

This may seem a long and roundabout way to get to where I was last Christmas, but I’m hoping that this represents the beginnings of a whole new way of working with the Raspberry Pi.

I’m excited, even if nobody else is…