• (I had intended a series of status posts from the beginning but that didn’t happen. Instead, I’ll start with current events and (perhaps) backfill later.)

Recent accomplishments - 2022-09

• Completed script interpreter to pre-segwit. Now I can (fetch and) validate all blocks and transactions (via their cryptographic hash linkage), verifying inputs and outputs via scripts - up until segwit. Next steps: Segwit changes.

• First transaction index on block/transaction storage. SQLite storage. (Not as transparent as I want so the second index will be plain ASCII row storage (but with a multi-level look-through structure like leveldb).)

• Conversion to message queuing backbone. Message queue is internal RAM-based and homegrown. And quite simple.

  Conversion was fairly easy as I had intended to go here all along. All data processing has been done in functional style^† with separate small procedures configured as a data pipeline. (Where parts of the pipeline are quite simple, though there are conditional branches, flow control branches, and joins, and (as yet) no loops). But the structure of the pipeline has been via explicit procedural methods (i.e., a function with a bunch of methods in a row: first do this, now do this, etc.) with data passed.

  Now, the pipelines are explicit: each data item has its recipe to be followed (attached to it as metadata) which directs the event queue where it should go next.

  This is necessary in order to:

  1. Be able to provide monitoring of the state of all the different data items in progress, and
  2. To allow (eventually) for a workflow manager/controller/orchestrator.

Next steps

• Segwit support.
• Add a multi-level pass-through cache layer to block storage so I can start putting blocks on different storage devices, and start sharing blocks between instances. At the same time, add an S3 block storage capability.

^† The main difference between “functional style” and, say, “object-oriented style”, from this point-of-view, is that functional style is based on a data pipeline where the focus is on the data which is passed from (pure) function to (pure) function, undergoing transformations. As opposed to the object-oriented style where the focus is on objects with behavior that “send messages” (i.e., pass data) from one to another in an orchestration so that each object executes the behavior of the messages it receives (in order!) by doing immediate calculations as well as passing messages to other objects.