WriteBufferProxy

From OpenCog

The WriteBufferProxyNode is an extension of the WriteThruProxy that buffers up write requests over a period of time, and consolidates repeated writes of the same Atom, so as to reduce the overall number of writes that get passed through. It is a write-cache; it de-duplicates repeated writes, before eventually passing them on to the final destination. Useful if the final target is slow; dramatic performance improvements can be gotten if the source is performing intensive, rapid updates of a small number of Atoms.

A small fraction of the buffer is written out over time, guaranteeing that writes do eventually occur. The write strategy is exponential decay: a fraction of the buffer is written out after seconds. The number is the characteristic decay constant of the buffer. By default, seconds. This is configurable. If/when the buffer gets small (because the incoming writes have tapered off), then the buffer is flushed in full. This is to avoid a long lingering tail of pending writes (That is, the long exponential tail is chopped off).

Example

The base usage is just like that for the WriteThruProxy. If only one write target is needed, then specify

  (ProxyParameters
     (WriteBufferProxy "write buffer")
     (RocksStorageNode "rocks:///tmp/foo.rdb"))

To actually use this, open the proxy just like any other StorageNode:

(cog-open (WriteBufferProxy "write buffer"))

The WriteBuffer handles writes only; it drops reads on the floor. To get a complete system, capable of handling reads and writes, a read source must be specified, with ReadWriteProxy. For example:

  (ProxyParameters
     (ReadWriteProxy "unbuff read, buff write")
     (List
        (RocksStorageNode "rocks:///tmp/foo.rdb")
        (WriteBufferProxy "write buffer")))

The first argument handles the reads, the second handles the writes. It is used normally, by saying

   (cog-open (ReadWriteProxy "unbuff read, buff write"))

See /examples/atomspace/persist-buffer.scm for a working example.

Time Constant

The buffer time constant can be set with a NumberNode:

  (ProxyParameters
     (WriteBufferProxy "write buffer")
     (RocksStorageNode "rocks:///tmp/foo.rdb")
     (Number 42))

The above sets the decay constant to 42 seconds; by default, it is 60 seconds.

Flush

A full flush of the buffer can be forced with the (barrier) command.

Stats

Performance statistics can be printed with

  (display (monitor-storage (WriteBufferProxy "write buffer")))

See also