SequentialReadProxy
The SequentialReadProxyNode is a ProxyNode that passes on read requests to a sequence of StorageNodes, until one of them responds with data. Reads are requests involving the reading of Atoms and Values. This includes `fetch-atom`, `fetch-value`, `fetch-incoming-set` and `fetch-incoming-by-type`. It can be used to build read-write overlays on top of large shared read-only datasets.
Example
Suppose you have two databases: one containing some huge amount of data, and a second, containing just deltas to the first. You would organize this so that the delta-database is read first, and, if an Atom is not found there, then the read is passed to the larger underlying database. This would be written as
(ProxyParameters
(SequentialReadProxy "overlayer")
(List
(RocksStorageNode "rocks:///home/me/mystuff/delta-overlay.rdb")
(RocksStorageNode "rocks:///usr/share/data/big-guy.rdb"))))
To actually use this, open the proxy just like any other StorageNode:
(cog-open (ReadThruProxy "overlayer"))
Then you can fetch data:
(fetch-atom (Concept "foo"))
which will fetch all of the Values on (Concept "foo"), assuming that it was in the database.
Large Read-only Datasets
A somewhat more practical example would be to make the overlay read-write, while the underlay is read-only. That way, updates won't trash the underlay. This mode is envisioned for large science datasets, e.g. genomic/proteomic datasets.
First, set up the read-chain:
(ProxyParameters
(SequentialReadProxy "overlay no. #42")
(List
(RocksStorageNode "rocks:///home/scientist-x/experiment-42.rdb")
(RocksStorageNode "rocks:///usr/share/data/local-genomics-dataset.rdb")
(CogStorageNode "cog://mozi-ai.com:17001"))))
This will do a three-deep lookup. First, a read attempt will be directed to experiment-42.rdb. If this fails to provide any Atoms, the read is passed on to local-genomics-dataset.rdb next. If this still doesn't provide an answer, then the network-remote dataset at mozi-ai.com is queried.
That handles the reads. What about writes? Try this:
(ProxyParameters
(ReadWriteProxy "delta 42")
(List
(SequentialReadProxy "overlay no. #42")
(RocksStorageNode "rocks:///home/scientist-x/experiment-42.rdb"))))
This will send reads to the Sequential proxy, while writes will always go to experiment-42.rdb and nowhere else.
