Packaging

From OpenCog
Jump to: navigation, search

The task breakdown list below represents a general order of how a packaging project should proceed. Some items can be worked on in parallel, however minimal buildbot builds should be occurring at every stage before fully moving on to subsequent items in the list. Many stages contain some elements of experimentation that will be made redundant in following steps (e.g. using different packing methods to achieve similar results).

Packaging Ontology Graph

Opencog-deps.png

Graph note: items in grey are not included in the packaging scheme, but shown for points of reference only.

Notes:

  • All scripts of the form ocscript are located at [1]
  • All new scripts, including Buildbot master.cfg and Windows PowerShell scripts, should be placed on Github
  1. DEBs, stand-alone
    1. coordinate on the OpenCog list how to best create deb-src packages from the opencog/opencog git repo. This will improve overall maintainability of OpenCog and may involve coordinating minor code refactoring.
    2. use a consensus packaging scheme, proposed in dotty file below (just a few example pre-packaged dependencies are shown in diamond boxes).
    3. see http://developer.ubuntu.com/packaging/html/debian-dir-overview.html
    4. extend 'ocpkg' with 'debs' package type which pulls control files from Launchpad bzr (after validation & testing these control files will be placed at lp:opencog/debian)
  2. DEBs, on Launchpad PPA
    1. see http://developer.ubuntu.com/packaging/html/udd-newpackage.html
    2. when complete, packaging should eliminate the need for the ocpkg script to install build dependencies. apt-get build-dep opencog or apt-get install opencog-dev should function in its place.
    3. new 'ocslave' script to create Buildbot slaves by installing buildbot, calling ocbootstrap (which calls ocpkg) and configuring & submitting slave details
    4. also possible should be full automation of building opencog images so they can be pulled and run on *any* Linux (and OSX via boot2docker, and Windows via Vagrant), e.g. docker run -d -p 17001:17001 -p 5000:5000 opencog/opencog
    5. requires maintaining Buildbot slaves to generate, install, unit & load test, and build from source DEBs
  3. Demo Installer for Windows
    1. extend 'ocpkg' with 'demovm' package type using VMBuilder with Unity3D proxy pre-configured (mostly to test OpenCog DEB package correctness, like default config settings, init scripts, etc.)
    2. new 'ocdemo_setup.ps1' PowerShell & chocolatey script to automate installation of demovm image + Portable-VirtualBox + Unity (Unity Technologies produce proprietary non-free software; it's a violation of their terms of service to redistribute the runtime without a 'Pro' license; however, downloads can be automated).
    3. extend 'ocslave' to create buildbot slaves to generate ocdemo_setup.ps1 + corresponding demovm image
    4. requires maintaining Buildbot slave to generate ocdemo_setup.ps1 scripts and corresponding demovm images built from the latest OpenCog source
  4. RPMs via OBS
    1. extend 'ocpkg' with 'obs' package type to automate submission of OpenCog source to OBS
    2. extend 'ocslave' to create Buildbot slaves to submit source, install, unit & load test RPMs
    3. requires maintaining Buildbot slaves, probably openSUSE, Fedora and CentOS at a minimum
  5. Cygwin port
    1. new 'ocygbootstrap' PowerShell script to automate installing Cygwin (setup.exe command line switches)
    2. extend 'ocpkg' to detect Cygwin for the 'local' (default) package type
    3. extend 'ocslave' to create [Buildbot slaves] to build from source on Windows+Cygwin
    4. requires maintaining a Windows Buildbot slave for this purpose
  6. Cygwin packaging
    1. extend 'ocpkg' with new 'cygpkg' type runnable on Windows+Cygwin
    2. extend 'ocslave' to create Buildbot slaves by calling ocygbootstrap to generate, install, unit & load tests Cygwin packages
    3. requires maintaining a Windows Buildbot slave and to install, unit & load tests Cygwin packages
  7. Demo Installer for Windows
    1. extend 'ocpkg' with 'cygdemo' package type runnable on Windows+Cygwin
    2. extend 'ocdemo_setup.ps1' to automate installation of Cygwin packages and Unity Player (ocdemo_setup with OpenCogVM becomes the fallback option if Cygwin build fails)
    3. requires maintaining Buildbot slave to generate ocdemo_setup.ps1 scripts

Miscellaneous

  1. See also: [David's reply] on mailing list for some thoughts related to packaging OpenCog.
  2. David, Nil and Amber agreed that following guidelines must be followed for ocpkg:
    1. The default behavior of running '$ocpkg' (with no parameters) should make it download minimal dependencies.
    2. ocpkg is also written for the systems administrator and with a few adaptations should be usable without a network connection. E.g. consider the case of a classroom with a spotty or limited internet connection, it should be easy for an administrator to quickly setup many machines without waiting for network downloads or timeouts, etc.
  3. Chocolatey and Vagrant cover all necessary VM and image automation on Windows.
digraph packages {
concentrate=true;
size="30,40";

"libatomspace" -> "libatomtable";
"libatomspace" -> "libatomredis" [color=grey66];
"libatomspace" -> "libatomcache" [color=grey66];
"CogServer" -> "libatomspace";
"CogServer" -> "libspatial";
"CogServer" -> "libcogutil";

"MOSES" -> "libcomboreduct";
#"MOSES" -> "libatomcache" [color=grey66];
"MOSES"-> "CogServer" [style=invis];
"PLN"-> "python2.7";
"PLN"-> "CogServer";
#"PLN" -> "libatomcache" [color=grey66];
"Fishgram" -> "CogServer";
"Fishgram" -> "python2.7";
"Attention" -> "CogServer";
"DeSTIN"-> "CogServer";
"Embodiment"-> "CogServer";
"OpenPsi" -> "CogServer";
"Visualization"-> "CogServer";

"libcomboreduct" [shape=box];
"python2.7" [shape=diamond];

"libatomcache" [shape=box] [color=grey66] [fontcolor=grey66];
"libatomredis" [shape=box] [color=grey66] [fontcolor=grey66];
"libatomtable" [shape=box];
"libatomspace" [shape=box];
"libspatial" [shape=box];
"libcogutil" [shape=box];
"CogServer" [shape=box] [label="\N\n opencog-server"];

"MOSES" [shape=ellipse] [label="\N\n opencog-moses \n (versioned)"];
"PLN" [shape=ellipse] [label="\N\n opencog-pln" ];
"Fishgram" [shape=ellipse] [label="\N\n opencog-fishgram" ];
"Attention" [shape=ellipse] [label="\N\n opencog-ecan" ];
"DeSTIN" [shape=ellipse] [label="\N\n opencog-destin" ];
"Embodiment" [shape=ellipse] [label="\N\n opencog-embodiment \n (versioned)"];
"OpenPsi" [shape=ellipse] [label="\N\n opencog-psi \n (versioned)"];
"Visualization" [shape=ellipse] [label="\N\n opencog-viz" ];

"RelEx" [shape=ellipse] [color=grey33] [fontcolor=grey33] [label="\N\n relex \n (versioned)" ];
"RelEx" -> "jre" [color=grey33];
"RelEx" -> "link-grammar" [color=grey33];
"jre" [shape=diamond] [color=grey33] [fontcolor=grey33] ;
"link-grammar" [shape=diamond] [color=grey33] [fontcolor=grey33] ;

"Embodiment" -> "Unity3D" [style=dashed] [color=grey33];
"Unity3D" [shape=house] [label="\N\n (closed source)"] [color=grey33] [fontcolor=grey33];

"OpenCog" [shape=plaintext] [label="\N\n opencog \n (metapackage)"];

"OpenCog" -> "MOSES";
"OpenCog" -> "PLN";
"OpenCog" -> "Fishgram";
"OpenCog" -> "Attention";
"OpenCog" -> "DeSTIN";
"OpenCog" -> "Embodiment" [style=invis];
"OpenCog" -> "OpenPsi";
"OpenCog" -> "Visualization";
"OpenCog" -> "RelEx" [style=invis];
}