OpenCog shell

From OpenCog

This page describes how to run the OpenCog network server. The server provides network connections, allowing either Scheme, Python or a limited range of JSON to be used over a network socket. Both telnet and WebSockets access methods are supported.

Examples on how to use subsystems, including the scheme shell, and a python shell, can be found in their respective pages.

The OpenCog shell is provided by the CogServer.

Starting the CogServer

The CogServer can be started in three different ways: from the scheme command line, from the python command line, and from the bash (unix) shell.

Prerequistes

It is assumed that you have already compiled and installed the CogServer. If not, this must be done first. Very briefly, this consists of the steps:

git clone https://github.com/opencog/cogserver
cd cogserver; mkdir build; cd build
cmake ..; make -j
sudo make install

From the bash prompt

The cogserver binary usually installs in /usr/local/bin; make sure this is in your path. Then just say

$ cogserver

You should see the following output:

Listening on port 17001
Listening on port 18080
...
...

The port at 17001 is the telnet port, the port at 18080 is the WebSockets port. Both of these can be changed; for example

$ cogserver -p 17002 -w 19042

From the guile prompt

To run it from guile, the cogserver module must be loaded:

guile> (use-modules (opencog) (opencog cogserver))
guile> (start-cogserver)

Note you can also stop it with (stop-cogserver). You can access documentation as:

guile> ,d start-cogserver

This page describes assorted parameters that can be set: the port numbers, the prompts, and more.

From the python prompt

Yes, this works. Someone should document it.

Connecting to the CogServer

The CogServer, by default, listens for telnet connections on port 17001. After starting the CogServer, use telnet to connect to it. From from a separate terminal window, run

telnet localhost 17001

Optionally, enable readline, which enables you to edit the shell's history run:

rlwrap telnet localhost 17001

This assumes that rlwrap is installed on your machine.

You should be presented with:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
opencog>

Run "help"<enter> to get a list of commands. After additional modules are loaded, additional commands will appear.

Available commands:
 config:    Config a loaded module
 help:      List the available commands; print help for a specific command
 json:      Enter the JSON shell
 list:      List the currently loaded cogserver modules
 load:      Load a cogserver module
 py:        Enter the python shell
 py-eval:   Run a block of python code, and return immediately
 quit:      Close the shell connection
 scm:       Enter the scheme shell
 sexpr:     Enter the s-expression shell
 shutdown:  Shut down the cogserver
 stats:     Print some diagnostic statistics about the server.
 top:       Print server stats continuously
 unload:    Unload an opencog module

To get information about any of the commands, use "help <name of command>". For example

opencog> help scm                                                              
Usage: scm [hush|quiet|sync]                                                   
                                                                              
Enter the scheme interpreter shell. This shell provides a rich                 
and easy-to-use environment for creating, deleting and manipulating            
OpenCog atoms and truth values. It provides a full R5RS-compliant              
interactive scheme shell, based on the GNU Guile extension language.           
                                                                              
If 'hush' or 'quiet' is specified after the command, then the prompt           
will not be returned.  This is nice when catting large scripts using           
netcat, as it avoids printing garbage when the scripts work well.              
If 'sync' is specified after the command, then the output is sync,             
instead of async.                                                              

Telnet, UTF-8 and Readline bugs

There are assorted bugs that can bite.

telnet vs UTF-8

Not all versions of telnet are UTF-8 clean. These version will handle many or most UTF-8 strings, but not all. For example, in buggy versions, (Concept "“") will cause a ABORT: decoding-error ... Get out of this with a ctrl-C, and use netcat, instead. The problem is described here, in reddit.

The work-around is to use netcat, which is 8-bit-clean. Like so:

 rlwrap nc localhost 17001

This works great, except that sending ctrl-C to the CogServer is now harder: you have to send ctrl-V ctrl-C because a naked ctrl-C will just kill netcat.

rlwrap eats prompts

Some versions of rlwrap eat the prompt. This is described in a rlwrap bug report. This happens on Deian stable, with libreadline8 version 8.1 and rlwrap version 0.43.

There is no known workaround. Just squint your eyes and pretend everything is OK.

WebSockets

You cannot connect to the WebSocket port with telnet; it won't work. You need a WebSocket client. Any modern web-broswer will do. For example, while the cogserver is running, click on this link: http://localhost:18080 To get to the json shell and to play with it, pull out your javascript skills, and visit the /examples page.

Loaded Modules

The list command will list the currently loaded modules. It should show something similar to this:

   Module Name           Library            Module Directory Path
   -----------           -------            ---------------------
BuiltinRequestsModule libbuiltinreqs.so  /usr/local/lib/opencog/modules
JsonShellModule       libjson-shell.so   /usr/local/lib/opencog/modules
PythonShellModule     libpy-shell.so     /usr/local/lib/opencog/modules
SchemeShellModule     libscheme-shell.so /usr/local/lib/opencog/modules
SexprShellModule      libsexpr-shell.so  /usr/local/lib/opencog/modules
TopShellModule        libtop-shell.so    /usr/local/lib/opencog/modules

These are always loaded by default. At this time, this is pretty much all of them. New modules are easy to create, but what would you need them for?

Monitoring the CogServer

The CogServer offers two commaonds for monitoring performance: stats and top. The topprints the same thing as stats, updating every 3 seconds. This interval can be adjusted; see help top. Example output:

stats
-----
02 Dec 00:24:53 2022 UTC ---- up-since: 02 Dec 00:19:16 2022
status: running  last: 02 Dec 00:19:26  tot-cnct:    1  port: 17001
max-open-socks: 10   cur-open-socks: 1   num-open-fds: 21  stalls: 0
cpu: 1.002 secs  user: 0.315  sys: 0.687     tot-lines: 50
maxrss: 59120 KB  majflt: 0  inblk: 0  outblk: 400

OPEN-DATE        THREAD  STATE NLINE  LAST-ACTIVITY  K U SHEL QZ E PENDG
02 Dec 00:19:26   411655 iwait     2 02 Dec 00:24:53 T 1 cogs           

The help stats command will explain all this:

The current date in UTC is printed, followed by:
 up-since: the date when the server was started.
 last: the date when the most recent connection was opened.
 tot-cnct: grand total number of network connections opened.
 cur-open-socks: number of currently open connections.
 num-open-fds: number of open file descriptors.
 stalls: times that open stalled due to hitting max-open-cnt.
 tot-lines: total number of newlines received by all shells.
 cpu user sys: number of CPU seconds used by server.
 maxrss: resident set size, in KB. Taken from `getrusage`.

The table shows a list of the currently open connections.
The table header has the following form:
OPEN-DATE THREAD STATE NLINE LAST-ACTIVITY K U SHEL QZ E PENDG
The columns are:
 OPEN-DATE -- when the connection was opened.
 THREAD -- the Linux thread-id, as printed by `ps -eLf`
 STATE -- several states possible; `iwait` means waiting for input.
 NLINE -- number of newlines received by the shell.
 LAST-ACTIVITY -- the last time anything was received.
 K -- socket kind. `T` for telnet, `W` for WebSocket.
 U -- use count. The number of active handlers for the socket.
 SHEL -- the current shell processor for the socket.
 QZ -- size of the unprocessed (pending) request queue.
 E -- `T` if the shell evaluator is running, else `F`.
 PENDG -- number of bytes of output not yet sent.


Monitoring the CogServer Log File

The best way to monitor the CogServer is to use the stats or the top command at the CogServer telnet shell. However, it also generates a logfile, which record startup information, as well as serious errors.

The CogServer will write log entries from various subsystems to the file /tmp/cogserver.log. Some subsystems may have their own log files; refer to their respective documentations for details.

To keep an eye on the log file, start a new Terminal window and run

tail -f /tmp/cogserver.log

You should see some output similar to the following:

opencog@locahost:/opencog/build$ tail -f /tmp/cogserver.log 
[2016-02-23 09:16:27:289] [INFO] Loaded /usr/local/share/opencog/scm/utilities.scm
[2016-02-23 09:16:27:295] [INFO] Loaded /usr/local/share/opencog/scm/apply.scm
[2016-02-23 09:16:27:300] [INFO] Loaded /usr/local/share/opencog/scm/file-utils.scm
[2016-02-23 09:16:27:300] [WARN] Failed to load file scm/persistence.scm: 2 No such file or directory
[2016-02-23 09:16:27:305] [INFO] Loaded /usr/local/share/opencog/scm/config.scm
[2016-02-23 09:16:27:317] [INFO] Loaded /usr/local/share/opencog/scm/repl-shell.scm
[2016-02-23 09:16:27:323] [INFO] Loaded /usr/local/share/opencog/scm/av-tv.scm
[2016-02-23 09:16:27:329] [INFO] Loaded /usr/local/share/opencog/scm/rule-engine-utils.scm
[2016-02-23 09:16:27:335] [INFO] Loaded ..//opencog/cogserver/scm/config.scm
[2016-02-23 09:16:27:336] [INFO] Starting CogServer loop.