GPI Framework Developer’s Guide

This guide introduces the various structures of the GPI framework, discusses the runtime entry points and covers some of the package dependencies for mastering a GPI distro. Its intended for GPI developers who want to make contributions to the framework in the areas of UI, deployment, runtime operation, etc… If your interested in extending the scientific algorithms used by the framework (i.e. creating new nodes), then checkout the Node Developer’s Guide as well the community page for listings of other node library projects

The Framework

The ‘framework’ and ‘core-node’ projects are separate entities. The framework provides the UI, runtime environment and build environment for the node libraries. This means the framework doesn’t provide any of the scientific algorithms itself, however, it pulls together some basic numeric packages (C++ and Python) to facilitate the development of nodes.

The repository that holds the framework project can be accessed here:

The framework project is organized in the following directory structure:

  • bin/
  • doc/
  • include/
  • launch/
  • lib/

The following sections introduce the software contained in each of these directories.


The ‘bin/’ directory holds the launching mechanisms for starting GPI (either as a GUI or a command-line utility), the command-line make, or running the updater. These mechanisms are accessed by the following scripts:

  • gpi_launch
  • gpi_make
  • gpi_update

The scripts are fairly simple, they may take command-line arguments and generally access a section of the main gpi library (discussed in the ‘lib’ section). Their purpose is to be configurable for a specific Anaconda Python install and make OS specific changes necessary to run GPI.

For example, the following code block is take from gpi_launch:

#!/usr/bin/env python


import sys, os

# Check for Anaconda PREFIX, or assume that THIS file location is the CWD.
GPI_PREFIX = '/opt/anaconda1anaconda2anaconda3' # ANACONDA
if GPI_PREFIX == '/opt/' + 'anaconda1anaconda2anaconda3':
    GPI_PREFIX, _ = os.path.split(os.path.dirname(os.path.realpath(__file__)))
    GPI_LIB_DIR = os.path.join(GPI_PREFIX, 'lib')
    if GPI_LIB_DIR not in sys.path:
        sys.path.insert(0, GPI_LIB_DIR)

# gpi
from gpi import launch

if __name__ == '__main__':

The shebang at the top specifies the first python instance in the user environment should be used to start this launching script. This means that you could download the framework project and run it against any Python installation, provided it has the necessary dependencies. The subsequent bit of logic determines whether the ‘conda’ package manager was used to install GPI; if it has, then the GPI_PREFIX will point to the location of the gpi library within the Anaconda Python installation. If ‘conda’ wasn’t used then the script uses dead reckoning to determine the location of the gpi library assuming the script was initiated within the framework directory structure. This is the same basic process in each script.


The ‘doc/’ directory contains the very documentation that you are reading now. It is written in reStructuredText and is compiled using the Sphinx documentation generator. This is auto-generated for each commit and hosted by the ReadTheDocs project.

To build these docs locally (if you intend to modify them), install Sphinx and simply run:

$ make html

in the ‘doc/’ directory. Then open the relevant ‘.html’ files that have been generated under the ‘doc/_build/’ directory.


The ‘include/’ directory contains the API code for a Python-C interface called PyFI. While the GPI UI doesn’t call on this code, it is provided as a portability layer for GPI nodes that depend on Python-C modules, written with PyFI C++ API. While you can still write GPI nodes with Python extension modules supported by SWIG or Boost these will be extra dependencies of your node library that will have to be communicated to your end users.

At the time PyFI was written, the aforementioned SWIG and Boost libraries didn’t yet have the capability to transfer numeric arrays between Python and C without copying the data. This was being developed in a project called Boost.NumPy, and is now part of the Boost.Python support package. PyFI also has the capability of allocating numeric arrays from Python, to be used in the embedded C routine, which circumvents the need to copy data between Python and C.

You can read more about PyFI in the Node Developer’s Guide:


The ‘launch/’ directory contains the GPI UI start-up scripts that meet the porcelain:

  • GPI.desktop
  • gpi.command

The GPI.desktop and scripts are converted to icon launchers for the Gnome (Ubuntu Linux) and MacOS/OSX desktops. They both eventually call on the gpi.command script which handles OS specific parameters for GPI. The launcher script’s link to the gpi.command script is not immediately obvious by inspecting these pieces of code, because there are path manipulations that happen in each of these scripts when they are part of a conda package deployment. To see how these scripts are placed in a deployment process, check out the conda deployment hook

As mentioned above, the gpi.command script provides some unique launching parameters depending on the OS. These differences are as follows:

In OSX, the main OS menu-bar will display the name of the binary being run. Since GPI is called as a library via Python, the Python binary is soft-linked in the system’s temp directory as “GPI” before calling it to start the GPI runtime. This will cause the OS menu-bar to display “GPI” in the upper left-hand corner.

In Ubuntu Linux, there have been specific versions of the desktop environment that cause Qt to default to one of the older style UI skins. To ensure that GPI is correctly started with the look and feel consistent to that of its OSX counterpart, the “cleanlooks” style is forced as a command-line option.


The ‘lib/’ directory contains the ‘gpi’ python library. This library is pure-Python and contains all the elements of the runtime environment. The section The ‘gpi’ Python Library catagorizes and lists each of the main object classes within the ‘gpi’ lib.

Building a Distro

The GPI distributions leverage the cloud for hosting GPI itself and some extra binary packages that are specifically configured for use with GPI. The supporting Anaconda-Cloud packages can be found at These packages were built using the scripts that can be found in the project. This project has a master build script (, that can be tailored to specific package needs such as platform dependence, Python version, release candidate, etc…

Since the gpi-framework project is pure python, it doesn’t have any platform specific building requirements. The conda package is simply a copy of the framework code with some specific launching details that are OS dependent (see the launch/ section above).

The astyle, eigen, fftw, and gpi-core-nodes packages are all C/C++ based and require platform specific compilation. This requires that the master build script is run on each type of target platform, with the necessary indicator options chosen. For example, if we want to build the framework and core-node packages, then we’d use a command like the following:

$ ./ --force-upload --auto-upload --channel gpi --tag main --python-version 35 --build-number 0 --package gpi-framework,gpi-core-nodes

This command would have to be run on both a Linux machine and an OSX machine so that Anaconda would have each platform specific version.

To ensure a GPI compatible environment, its common to use the installer script ( to auto-install a local copy of Miniconda to run these build scripts against. This can be done on Linux and OSX.


To create the OSX App distro there there is an additional project, for generically wrapping Miniconda based applications in OSX, called Wr[App]-A-Conda. The App can also be bundled in a .dmg file by using the create-dmg project.

These two external application packaging projects (‘Wr[App]-A-Conda’ and ‘create-dmg’) are automatically invoked by the script found in the osx_stack section of the project (provided that these two projects are in your PATH environment).

The ‘gpi’ Python Library

The gpi Python library is collection of inherited PyQt classes (for the UI), Numpy data handling libs, configuration, command-line and build system interfaces. The following sections will introduce the sub-library components within these contexts.

GUI (PyQt Classes)

The gpi modules responsible for the canvas, node menu and other dialogues are as follows:


    • class gpi.canvasGraph.GraphWidget(title, parent)[source]

      Provides the main canvas widget and background painting as well as the execution model for the canvas.


Figure 1 - The finite state machine model for the canvas execution.


    • class gpi.canvasScene.CanvasScene(parent=None)[source]

      Supports the main canvas widget by drawing shapes for displaying interactions between elements on the canvas (e.g. selecting nodes).


    • class gpi.mainWindow.MainCanvas[source]
      • Implements the canvas QWidgets, contains the main menus and provides user settings via menu or rc file.
      • Anchors the canvas and provides the main menu and status bar.

    • class gpi.edge.Edge(sourcePort, destPort)[source]

      Provides the connection graphic and logic for nodes. -No enforcement, just methods to retrieve connected nodes.

    • class gpi.edge.EdgeTracer(graph, destPort, sourcePort)[source]

      When an edge is deleted it will be replaced with this static object for a few seconds and then remove itself.


    • class gpi.launch.Splash(image_path)[source]

      The splash screen that appears at GPI launch. This contains a copy of the boilerplate required by Dignity Health.

    • gpi.launch.launch()[source]

      Starts the main application loop, parses any user config and commandline args.


    • class gpi.layoutWindow.LayoutWindow(graph, layoutType, label, parent=None)[source]

      This class controls the low level operations on “Layout Windows”. It handles the drag and drop events for pulling widgets out of “Node Menus”. It also handles low-level serialization.

    • class gpi.layoutWindow.LayoutMaster(graph, config=0, macro=False, labelWin=False, parent=None)[source]

      This is the main “Layout-Window” API. It controls the layout config, reinstantiation of layouts, opening and closing, etc…


    • class gpi.library.FauxMenu(realMenu, menuPos, parent=None)[source]

      For the node search in the right-button mouse menu. This label is what is rendered during the intermediate searching.

    • class gpi.library.NodeCatalogItem(fullpath)[source]

      A single entry to a Node database. For information such as library, path, type, etc…

    • class gpi.library.NetworkCatalogItem(fullpath)[source]

      A single database entry to a GPI Network database.

    • class gpi.library.Library(parent)[source]

      Contains all the Node, Network, and GPIType path searching, mouse menu generation and indexing for the node library. The contents of the library are loaded at startup (each time) and when the user adds Nodes via drag’n drop or menu contexts.


    • class gpi.macroNode.EdgeNode(portNum, parentItem)[source]

      The EdgeNode simply forwards port connections. It can take one OutPort connection, and multiple InPort connections. InPort tooltips and enforcement are concatenated and forwarded.

    • class gpi.macroNode.MacroAPI(node)[source]

      A class that has default operations that stub-out the NodeAPI.

    • class gpi.macroNode.PortEdge(macroParent, graph, menu=None, nodeIF=None, nodeIFscroll=None, role=None)[source]

      The PortEdge holds EdgeNodes for the MacroNode (they are the two node like pieces that bound the macro and the single collapsed macro representation).

    • class gpi.macroNode.MacroNodeEdge(source, dest)[source]

      Provides the connection graphic and logic for nodes. -No enforcement, just methods to retrieve connected nodes.

    • class gpi.macroNode.MacroNode(graph, pos)[source]

      The MacroNode menu is a stripped down version of the Node class with the capability of using a “Layout Window” instead of a NodeMenu.


    • class gpi.node.TimerPack[source]

      GUI updates often need to be prodded at some interval for a duration, and then shutdown. This seems to require 2 timers, one for interval, one for ON duration.

    • class gpi.node.EventManager[source]

      Stores new events (without duplicates) in a timely manner.

    • class gpi.node.NodeSignalMediator[source]

      A hack to add PyQt signals to a non QObject derivative.

    • class gpi.node.NodeAppearance[source]

      This class may be used to define node color, height, width, margins,etc..

    • class gpi.node.Node(CanvasBackend, nodeCatItem=None, nodeIF=None, nodeIFscroll=None, nodeMenuClass=None)[source]

      The graphics and execution manager for individual nodes.


Figure 2 - The finite state machine model for the node execution.


    • class gpi.nodeAPI.NodeAPI(node)[source]

      Base class for all external nodes.

      External nodes implement the extensible methods of this class: i.e. compute(), validate(), execType(), etc… to create a unique Node module.


      Initialize the node UI (Node Menu).

      This method is intended to be reimplemented by the external node developer. This is where Port and Widget objects are added and initialized. This method is called whenever a node is added to the canvas.

      See Adding Widgets and Adding Ports for more detail.


      The pre-compute validation step.

      This function is intended to be reimplemented by the external node developer. Here the developer can access widget and port data (see Accessing Widget Data and Accessing Port Data) to perform validation checks before compute() is called.

      Returns:0: The node successfully passed validation

      1: The node failed validation, compute will not be called and the canvas will be paused

      Return type:An integer corresponding to the result of the validation

      The module compute routine.

      This function is intended to be reimplemented by the external node developer. This is where the main computation of the node is performed. The developer has full access to the widget and port data (see Accessing Widget Data and Accessing Port Data).

      Returns:0: Compute completed successfully

      1: Compute failed in some way, the canvas will be paused

      Return type:An integer corresponding to the result of the computation

    • class gpi.port.Port(nodeWidget, CanvasBackend, portTitle, portNum, intype=None, dtype=None, ndim=None, menuWidget=None)[source]

      The base-class of the Node InPorts and OutPorts. This is responsible for Tool tips, serialization, drawing, painting, and type matching for connectivity.

    • class gpi.port.InPort(nodeWidget, CanvasBackend, title, portNum, intype=None, dtype=None, ndim=None, obligation=100, menuWidget=None, cyclic=False)[source]

      Defines the specific behavior for connecting to inports i.e. how to check for upstream data types, and obligation.

    • class gpi.port.OutPort(nodeWidget, CanvasBackend, title, portNum, intype=None, dtype=None, ndim=None, obligation=None, menuWidget=None)[source]

      Defines the specific behavior for connecting to outports i.e. how to check for downstream data types for scaling downstream ports on the canvas to highlight potentially valid connections.


    • class gpi.runnable.Runnable(func)[source]

      A helper class to offload I/O bound operations so that the UI can continue to function and redraw.

    • gpi.runnable.ExecRunnable(runnable)[source]

      A quick way to spawn a thread for function objects and bound methods




    • class gpi.update.JSONStreamLoads(in_str, linefeed=True)[source]

      Load multiple json objects from string. Returns loaded objects in a list.

    • class gpi.update.CondaUpdater(conda_prefix='/home/docs/checkouts/', dry_run=False)[source]

      Handles the underlying communication with the ‘conda’ program.

    • class gpi.update.UpdateWindow(dry_run=False)[source]

      A simple UI to display the GPI update process as its happening.


    • This file contains all of the Q-Widgets that have been wrapped and simplified for the GPI interface and Node development API.

    • class gpi.widgets.GenericWidgetGroup(title, parent=None)[source]

      This is the base-class for all widgets. It provides abstract methods and default behavior. From the node-developer’s perspective, this provides the widget-port, visibility, and collapsibility options.

Node/Canvas Execution


    • class gpi.functor.GPIFunctor(node, parent=None)[source]

      A common parent API for each execution type (i.e. ATask, PTask, TTask). Handles the data communications to and from each task type.

    • class gpi.functor.ATask(func, title, label, proxy)[source]

      App-Loop or Main-Loop executed task. This will block GUI updates. Data is communicated directly.

      NOTE: The apploop-type blocks until finished, obviating the need for signals or timer checks

    • class gpi.functor.PTask(func, title, label, proxy)[source]

      A forked process node task. Memmaps are used to communicate data.

      NOTE: The process-type has to be checked periodically to see if its alive, from the spawning process.

    • class gpi.functor.TTask(func, title, label, proxy)[source]

      A QThread based node runner. Data is communicated directly.

      NOTE: The thread-type emits a signal when its finished:

      gpi.Signal.finished() gpi.Signal.terminated()

  • (PyQt classes)

    • class gpi.nodeQueue.GPINodeQueue(parent=None)[source]

      The list of nodes to process based on UI and hierarchy changes.

  • (PyQt classes)

    • class gpi.stateMachine.GPIState(name, func, machine=None, efunc=None)[source]

      A single FSM state to be used with the GPI_FSM class. It manages its transitions, entry and exit functions.

    • class gpi.stateMachine.GPI_FSM(name='')[source]

      The GPI Canvase and Nodes operate within different states. The finite state machine allows for fast and easy checking to determine whether the user has performed an invalid operation or not.


    • gpi.topsort.topological_sort(items, partial_order)[source]

      Perform topological sort. items is a list of items to be sorted. partial_order is a list of pairs. If pair (a,b) is in it, it means that item a should appear before item b. Returns a list of the items in one of the possible orders, or None if partial_order contains a loop.

    • gpi.topsort.topsort(connection_list)[source]

      NRZ: The connection list is a list of pairs indicating flow direction through node ordering.

      -we only care about nodes with connections

      -so only itemize those nodes

Data Types & Communication

Most of the data communication between nodes is handled by the ‘functor’ module. The data type descriptions are all loaded at runtime by plugging everything under the ‘types’ directory.


    • This module is an extension for handling specific data types such as Numpy-arrays.

    • class gpi.dataproxy.DataProxy[source]

      Holds all file descriptor information for any object that is serializeable. The method functions facilitate serialization and deserialization on either side of the Proxy-Manager.


      Numpy arrays must be segments that are smaller than 2GiB (2^31 bytes).


      MMAP file descriptors are passed through the proxy only if there are enough available resources (i.e. rlimit).


    • This module facilitates the manipulation of GL objects by maintaining a list of GL calls to be supplied to a GL renderer. These objects are pure python and can therefore be easily serialized for passing between Nodes.


    • class gpi.types.globjectlist_GPITYPE.GLOList[source]

      Allows passing GPI-GL object definitions to be passed in lists.


    • class gpi.types.numpy_GPITYPE.NPYarray[source]

      The NPYarray type provides port enforcement for numpy multidimensional arrays (ndarray). The enforcement parms are type (ndarray), dtype, ndim, dimension range (drange), shape, and vec (the len of the last dim). Enforcement priority for aliased parms goes ndim->drange->shape. Although shape and vec can overlap, they are enforced independently.


    • class gpi.types.python_GPITYPE.INT[source]

      Enforcement for the standard python-int.

    • class gpi.types.python_GPITYPE.FLOAT[source]

      Enforcement for the standard python-float.

    • class gpi.types.python_GPITYPE.LONG[source]

      Enforcement for the standard python-long.

    • class gpi.types.python_GPITYPE.COMPLEX[source]

      Enforcement for the standard python-complex.

    • class gpi.types.python_GPITYPE.STRING[source]

      Enforcement for the standard python-str.

    • class gpi.types.python_GPITYPE.LIST[source]

      Enforcement for the standard python-list.

    • class gpi.types.python_GPITYPE.TUPLE[source]

      Enforcement for the standard python-tuple.

    • class gpi.types.python_GPITYPE.DICT[source]

      Enforcement for the standard python-dict.


    • class gpi.defaultTypes.GPIDefaultType[source]

      This default class is the base class of all GPITypes. It provides default behavior for edgeTips and toolTips. The default behavior for port and data type matching is to pass.

Loading & Storing Node/Types/Widgets


    A pymod loader for managing loaded nodes, types, and widgets.

    • gpi.loader.consolidatePaths(plist)[source]

      Take the set() of all abspaths(). Doesn’t guarantee original order.

    • gpi.loader.PKGroot(fullpath)[source]

      See if the file or directory pointed to by fullpath is a package dir or subpackage dir. Return the highest package dir.

    • gpi.loader.loadMod(fullpath)[source]

      Load modules .py or .pyc from the given path and store in sys.modules using the fullpath as the key. This will allow all plugins and node descriptions to be unique, even if they have the same name.

    • gpi.loader.findAndLoadMod(name, path=None, store_name=None)[source]

      Load modules .py or .pyc from sys.path or path, if given. ‘path’ must be a list.


      -not sure when this stopped being used or if find_module() affords us anything over the loadMod().


    • class gpi.catalog.CatalogObj[source]

      An abstract object to be used with the Catalog indexing object.

    • class gpi.catalog.Catalog[source]

      A generic catalog object for referencing/indexing/sorting a list of descriptions based on their content. Mostly just an extra interface to a dictionary that allows information merges. Keys are determined by the appended object.

    • Also see the ‘library’ module for more information on storing these types.

Loading & Storing Networks


    • Classes for loading and saving each version of gpi networks. Each class corresponds to a particular network file description. Upon load, the network file type is determined and then the appropriate class is installed. On write, the latest network description is used.

    • class, contents=None)[source]

      All networks should implement this base class and be named with the ‘Network_v<version>’ where ‘version’ is a str(integer).

    • class, contents=None)[source]

      The first version of the network interface compatible with the released GPI-beta. The file format is pickled list() and dict() objects. The original released header doesn’t necessarily contain a version number so network files w/o are assumed to be this version.

    • class, contents=None)[source]

      The second version of the network interface. This version supports new 2-level library scope which exists as an extra tag in the node settings dict. The file format is pickled list() and dict() objects. The original released header doesn’t necessarily contain a version number so network files w/o are assumed to be this version.

    • class, contents=None)[source]

      The third version of the network interface. This version supports the json data file format for better multi-platform support.

    • class[source]

      Determines the version of the file to be read, then chooses the correct loading object for that version. The loading object must translate the file contents to the various object settings that make up the network description. This object should handle the parent calls.

Compiling PyFI Modules


    • Use python distutils to build extension modules. This script can be called directly from the commandline to build C-extensions or check pure python extensions.

      A C/C++ extension module that implements an alorithm or method.

      To make, issue the following command:

      $ ./ <basename> or $ ./ <basename>.cpp or $ ./ <basename>.py

    • gpi.make.make(GPI_PREFIX=None)[source]

      Commandline interface to the make utilities.

Command Line Interface


    • class gpi.cmd.CmdParser[source]

      An object to parse input commandline args after the QApplication has done its own parsing.

User Configuration


    • class gpi.config.ConfigManager[source]

      An object that can load and generate the gpi config file. The gpi config file can potentially hold configs for:

      library paths, network dir path, data dir path, plugin paths, filetype-node associations, canvas window start size, UI style



    • class gpi.nodeTemplate_GPI.ExternalNode(node)[source]

      This is a GPI node template.

      You should replace this docstring with accurate and thorough documentation. INPUT:

      in - a numpy array


      out - a numpy array


      foo - an integer

BORG: Building Outside Relationships with GPI

The interface for easily encapsulating/assimilating external command-line programs is internally called “external binary encapsulation” or ‘ebe’.


    • class gpi.ebe.FilePath(wfunc=None, wdata=None, path=None, filename=None, suffix=None, nodeid=None, rfunc=None, asuffix=[])[source]

      Generate a tempfile-name and path based on THIS object’s id. If THIS object looses its reference then make sure the associated file is also deleted. The supplied read/writer functions can be used to write and retrieve the file information. If a tempfile-path and name is all that is needed, then this object can be instantiated without any arguments.

      path: /tmp (default GPI tmp dir) filename: additional to the nodeid suffix: additional to the nodeid (i.e. ‘.jpg’) nodeid: node’s location in memory (id()) rfunc: reader function with footprint:

      data = rfunc(‘filename’)

      wfunc: writer function with footprint:

      retcode = wfunc(‘filename’, data) retcode: None or 0 for success

      If no names are specified then THIS object id is used.

    • class gpi.ebe.Command(*args, **kwargs)[source]

      This object simplifies the situation where an external program generates a file and potentially takes a file as input. These files need to be communicated as commandline arguments, and also need to be read and written from GPI.

      in1 = FilePath(‘.cfl’, writer, data) out1 = FilePath(‘.cfl’, reader)

      # run command immediatly Command(‘fft’, in1, ‘-o’, out1, ‘-d1’)

      # setup a command list c = Command() c.arg(‘fft’) c.arg(in1, ‘-o’, out1) c.arg(‘-d1’)

      data =

Misc Features


    • A class for getting relevant system specifications. These can be used to display system information in the status bar, convey relative performance info in networks, etc…


    • This is an initial attempt at logging GPI sessions. Currently all logging information is printed to stdout b/c there is a missing mechanism to communicate logging statements back from forked processes.


    • File-node associations for data drag’n drop.


    • Miscellaneous constants used throughout GPI. These could be for z-buffer, internal type conventions, uri conventions, etc…

  • (PyQt classes)

    • class gpi.console.Tee(stdIO=None, parent=None)[source]

      (Mostly Unused) An attempt at providing console output to a GPI internal console window. This is part of an unfinished feature that is meant to give the user easy access to logging information.


    • class[source]

      (Deprecated) For grabbing the Node documentation for of each node found in all connected libraries. This is used for listing all the nodes available.

    • class[source]

      (Deprecated) For gathering all the relevant API documentation used in Node development.