fsleyes.gl

This package contains the OpenGL data and rendering stuff for FSLeyes. On-screen and off-screen rendering is supported, and two OpenGL versions (1.4 and 2.1) are supported. The contents of this package can be broadly categorised into the following:

• Canvases: A canvas is a thing that can be drawn on.

• Objects: An object is a thing which can be drawn on a canvas.

Quick start

import fsleyes.gl                 as fslgl
import fsleyes.gl.wxglslicecanvas as slicecanvas

# This function will be called when
# the GL context is ready to be used.
def ready():

    # The fsleyes.gl package needs to do
    # some initialisation that can only
    # be performed once a GL context has
    # been created.
    fslgl.bootstrap()

    # Once a GL context has been created,
    # you can do stuff! The SliceCanvas
    # will take care of creating and
    # managing GLObjects for each overlay
    # in the overlay list.
    canvas = slicecanvas.WXGLSliceCanvas(parent, overlayList, displayCtx)

# Create a GL context, and tell it to
# call our function when it is ready.
fslgl.getGLContext(ready=ready)

Canvases

A canvas is the destination for an OpenGL scene render. The following canvases are defined in the gl package:

SliceCanvas	The SliceCanvas represents a canvas which may be used to display a single 2D slice from a collection of 3D overlays.
LightBoxCanvas	The LightBoxCanvas represents an OpenGL canvas which displays multiple slices from a collection of 3D overlays.
Scene3DCanvas
ColourBarCanvas	Contains logic to render a colour bar as an OpenGL texture.

These classes are not intended to be used directly. This is because the gl package has been written to support two primary use-cases:

• On-screen display of a scene using a wx.glcanvas.GLCanvas canvas.

• Off-screen rendering of a scene to a file.

Because of this, the canvas classes listed above are not dependent upon the OpenGL environment in which they are used (i.e. on-screen or off-screen). Instead, two base classes are provided for each of the use-cases:

WXGLCanvasTarget	Base class for wx.glcanvas.GLCanvas objects.
OffScreenCanvasTarget	Base class for canvas objects which support off-screen rendering.

And the following sub-classes are defined, providing use-case specific implementations for each of the available canvases:

WXGLSliceCanvas	The WXGLSliceCanvas is a SliceCanvas, a wx.glcanvas.GLCanvas and a WXGLCanvasTarget.
WXGLLightBoxCanvas	The WXGLLightBoxCanvas is a LightBoxCanvas, a wx.glcanvas.GLCanvas and a WXGLCanvasTarget.
WXGLScene3DCanvas	The WXGLScene3DCanvas is a Scene3DCanvas, a wx.glcanvas.GLCanvas and a WXGLCanvasTarget.
WXGLColourBarCanvas	The WXGLColourBarCanvas is a ColourBarCanvas, a wx.glcanvas.GLCanvas and a WXGLCanvasTarget.
OffScreenSliceCanvas	The OffScreenSliceCanvas is a SliceCanvas which uses a RenderTexture as its target, for static off-screen OpenGL rendering.
OffScreenLightBoxCanvas	The OffScreenLightBoxCanvas is a LightBoxCanvas which uses a RenderTexture as its target, for static off-screen Open GL rendering.
OffScreenScene3DCanvas	The OffScreensScene3DCanvas is a Scene3DCanvas which uses a RenderTexture as its target, for static off-screen OpenGL rendering.
OffScreenColourBarCanvas	The OffScreenColourBarCanvas is a ColourBarCanvas which uses a RenderTexture for static off-screen OpenGL rendering.

The classes listed above are the ones which are intended to be instantiated and used by application code.

gl objects

With the exception of the ColourBarCanvas, everything that is drawn on a canvas derives from the GLObject base class. A GLObject manages the underlying data structures, GL resources (e.g. shaders and textures), and rendering routines required to draw an object, in 2D or 3D, to a canvas. The following GLObject sub-classes correspond to each of the possible types (the Display.overlayType property) that an overlay can be displayed as:

GLVolume(image, overlayList, displayCtx, …)	The GLVolume class is a GLImageObject which encapsulates the data and logic required to render Image overlays in 2D and 3D.
GLRGBVolume(image, overlayList, displayCtx, …)	The GLRGBVolume class is used to render RGB(A) Image overlays.
GLComplex(image, overlayList, displayCtx, …)	The GLComplex class is a sub-class of GLVolume, specialised for displaying Image overlays with a complex data type.
GLMask(image, overlayList, displayCtx, …)	The GLMask class encapsulates logic to render an Image instance as a binary mask in OpenGL.
GLLabel(image, overlayList, displayCtx, …)	The GLLabel class is a GLImageObject which encapsulates the logic required to render an Image overlay as a label image.
GLLineVector(image, overlayList, displayCtx, …)	The GLLineVector class encapsulates the logic required to render an Image instance of shape x*y*z*3, or type NIFTI_TYPE_RGB24, as a vector image, where the vector at each voxel is drawn as a line, and coloured in the same way that voxels in the GLRGBVector are coloured.
GLRGBVector(image, overlayList, displayCtx, …)	The GLRGBVector class encapsulates the logic required to render a x*y*z*3 Image instance as a vector image, where the direction of the vector at each voxel is represented by a combination of three colours (one colour per axis).
GLMesh(overlay, overlayList, displayCtx, …)	The GLMesh class is a GLObject which encapsulates the logic required to draw 2D slices, and 3D renderings, of a Mesh overlay.
GLMIP(image, overlayList, displayCtx, …)	The GLMIP class is a GLImageObject which can be used to render maximum-intensity-projections of an Image overlay onto a 2D canvas.
GLTensor(image, overlayList, displayCtx, …)	The GLTensor class encapsulates the logic required to render TensorImage overlays.
GLSH(image, overlayList, displayCtx, canvas, …)	The GLSH class is a GLVectorBase for rendering Image overlays which contain spherical harmonic (SH) coefficients that represent fibre orientation distributions (FOD).

These objects are created and destroyed automatically by the canvas classes instances, so application code does not need to worry about them too much.

Annotations

Canvases can be annotated in a few ways, by use of the Annotations class. An Annotations object allows lines, rectangles, and other simple shapes to be rendered on top of the GLObject renderings which represent the overlays in the OverlayList. The Annotations object for a canvas instance can be accessed through its getAnnotations method.

OpenGL versions and bootstrapping

FSLeyes needs to be able to run in restricted environments, such as within a VNC session, and over SSH. In such environments the available OpenGL version could be quite old, so the gl package has been written to support an environment as old as OpenGL 1.4.

The available OpenGL API version can only be determined once an OpenGL context has been created, and a display is available for rendering. The package-level getGLContext() function allows a context to be created.

The data structures and rendering logic for some GLObject classes differs depending on the OpenGL version that is available. Therefore, the code for these GLObject classes may be duplicated, with one version for OpenGL 1.4, and another version for OpenGL 2.1. The GLObject code which targets a specific OpenGL version lives within either the gl14 or gl21 sub-packages.

Because of this, the package-level bootstrap() function must be called before any GLObject instances are created, but after a GL context has been created.

fsleyes.gl.bootstrap(glVersion=None)

Imports modules appropriate to the specified OpenGL version.

The available OpenGL API version can only be queried once an OpenGL context is created, and a canvas is available to draw on. This makes things a bit complicated, because it means that we are only able to choose how to draw things when we actually need to draw them.

This function should be called after an OpenGL context has been created, and a canvas is available for drawing, but before any attempt to draw anything. It will figure out which version-dependent package needs to be loaded, and will attach all of the modules contained in said package to the gl package. The version-independent modules may then simply access these version-dependent modules through this module.

After the boostrap() function has been called, the following package-level attributes will be available on the gl package:

GL_VERSION	A string containing the target OpenGL version, in the format major.minor, e.g. 2.1.
GL_RENDERER	A string containing the name of the OpenGL renderer.
glvolume_funcs	The version-specific module containing functions for rendering GLVolume instances.
glrgbvolume_funcs	The version-specific module containing functions for rendering GLRGBVolume instances.
glrgbvector_funcs	The version-specific module containing functions for rendering GLRGBVector instances.
gllinevector_funcs	The version-specific module containing functions for rendering GLLineVector instances.
glmesh_funcs	The version-specific module containing functions for rendering GLMesh instances.
glmask_funcs	The version-specific module containing functions for rendering GLMask instances.
gllabel_funcs	The version-specific module containing functions for rendering GLLabel instances.
gltensor_funcs	The version-specific module containing functions for rendering GLTensor instances.
glsh_funcs	The version-specific module containing functions for rendering GLSH instances.
glmip_funcs	The version-specific module containing functions for rendering GLMIP instances.

This function also sets the Platform.glVersion and Platform.glRenderer properties of the fsl.utils.platform.platform instance.

Parameters

glVersion – A tuple containing the desired (major, minor) OpenGL API version to use. If None, the best possible API version will be used.

fsleyes.gl.getGLContext(**kwargs)

Create and return a GL context object for on- or off-screen OpenGL rendering.

If a context object has already been created, it is returned. Otherwise, one is created and returned.

See the GLContext class for details on the arguments.

Warning

Use the ready argument to GLContext.__init__(), and don’t call bootstrap() until it has been called!

class fsleyes.gl.GLContext(offscreen=False, other=None, target=None, createApp=False, ready=None, raiseErrors=False)

Bases: object

The GLContext class manages the creation of, and access to, an OpenGL context. This class abstracts away the differences between creation of on-screen and off-screen rendering contexts. It contains a single method, setTarget(), which may be used to set a WXGLCanvasTarget or an OffScreenCanvasTarget as the GL rendering target.

On-screen rendering is performed via the wx.GLCanvas.GLContext context, whereas off-screen rendering is performed via OpenGL.raw.osmesa.mesa (OSMesa is assumed to be available).

If it is possible to do so, a wx.glcanvas.GLContext will be created, even if an off-screen context has been requested. This is because using the native graphics card is nearly always preferable to using OSMesa.

Creating an on-screen GL context

A wx.glcanvas.GLContext may only be created once a wx.glcanvas.GLCanvas has been created, and is visible on screen. The GLContext class therefore creates a dummy wx.Frame and GLCanvas, and displays it, before creating the wx GL context.

Because wx contexts may be used even when an off-screen rendering context has been requested, the GLContext class has the ability to create and run a temporary wx.App, on which the canvas and context creation process is executed. This horrible ability is necessary, because a wx.GLContext must be created on a wx application loop. We cannot otherwise guarantee that the wx.GLCanvas will be visible before the wx.GLContext is created.

The above issue has the effect that the real underlying wx.GLContext may only be created after the GLContext.__init__ method has returned. Therefore, you must use the ready callback function if you are creating a wx GL context - this function will be called when the GLContext is ready to be used.

You can get away without using the ready callback in the following situations:

• When you are 100% sure that you will be using OSMesa.

• When there is not (and never will be) a wx.MainLoop running, and you pass in createApp=True.

__init__(offscreen=False, other=None, target=None, createApp=False, ready=None, raiseErrors=False)

Create a GLContext.

Parameters

• offscreen – On-screen or off-screen context?

• other – Another GLContext instance with which GL state should be shared.

• target – If other is not None, this must be a reference to a WXGLCanvasTarget, the rendering target for the new context.

• createApp – If True, and if possible, this GLContext will create and run a wx.App so that it can create a wx.glcanvas.GLContext.

• ready – Function which will be called when the context has been created and is ready to use.

Are raiseErrors

Defaults to False. If True, and if the ready function raises an error, that error is not caught.

setTarget(target)

Set the given WXGLCanvasTarget or OffScreenCanvasTarget as the target for GL rendering with this context.

_GLContext__createOSMesaContext()

Creates an OSMesa context, assigning it to an attribute called __context.

_GLContext__createWXGLCanvas()

Create a dummy wx.glcanvas.GLCanvas instance which is to be used to create a context. Assigns the canvas to an attributed called __canvas.

_GLContext__createWXGLContext(other=None, target=None)

Creates a wx.glcanvas.GLContext object, assigning it to an attribute called __context. Assumes that a wx.glcanvas.GLCanvas has already been created.

Parameters

• other – Another wx.glcanvas.GLContext` instance with which the new context should share GL state.

• target – If other is not None, this must be a wx.glcanvas.GLCanvas, the rendering target for the new context.

Warning

This method must be called via the wx.MainLoop.

_GLContext__createWXGLParent()

Create a dummy wx.Frame to be used as the parent for the dummy wx.glcanvas.GLCanvas.

__dict__ = mappingproxy({'__module__': 'fsleyes.gl', '__doc__': 'The ``GLContext`` class manages the creation of, and access to, an\n OpenGL context. This class abstracts away the differences between\n creation of on-screen and off-screen rendering contexts.\n It contains a single method, :meth:`setTarget`, which may\n be used to set a :class:`.WXGLCanvasTarget` or an\n :class:`OffScreenCanvasTarget` as the GL rendering target.\n\n\n On-screen rendering is performed via the ``wx.GLCanvas.GLContext``\n context, whereas off-screen rendering is performed via\n ``OpenGL.raw.osmesa.mesa`` (OSMesa is assumed to be available).\n\n\n If it is possible to do so, a ``wx.glcanvas.GLContext`` will be created,\n even if an off-screen context has been requested. This is because\n using the native graphics card is nearly always preferable to using\n OSMesa.\n\n\n *Creating an on-screen GL context*\n\n\n A ``wx.glcanvas.GLContext`` may only be created once a\n ``wx.glcanvas.GLCanvas`` has been created, and is visible on screen. The\n ``GLContext`` class therefore creates a dummy ``wx.Frame`` and\n ``GLCanvas``, and displays it, before creating the ``wx`` GL context.\n\n\n Because ``wx`` contexts may be used even when an off-screen rendering\n context has been requested, the ``GLContext`` class has the ability to\n create and run a temporary ``wx.App``, on which the canvas and context\n creation process is executed. This horrible ability is necessary, because\n a ``wx.GLContext`` must be created on a ``wx`` application loop. We cannot\n otherwise guarantee that the ``wx.GLCanvas`` will be visible before the\n ``wx.GLContext`` is created.\n\n\n The above issue has the effect that the real underlying ``wx.GLContext``\n may only be created after the ``GLContext.__init__`` method has returned.\n Therefore, you must use the ``ready`` callback function if you are\n creating a ``wx`` GL context - this function will be called when the\n ``GLContext`` is ready to be used.\n\n\n You can get away without using the ``ready`` callback in the following\n situations:\n\n - When you are 100% sure that you will be using OSMesa.\n\n - When there is not (and never will be) a ``wx.MainLoop`` running, and\n you pass in ``createApp=True``.\n ', '__init__': <function GLContext.__init__>, 'setTarget': <function GLContext.setTarget>, '_GLContext__createWXGLParent': <function GLContext.__createWXGLParent>, '_GLContext__createWXGLCanvas': <function GLContext.__createWXGLCanvas>, '_GLContext__createWXGLContext': <function GLContext.__createWXGLContext>, '_GLContext__createOSMesaContext': <function GLContext.__createOSMesaContext>, '__dict__': <attribute '__dict__' of 'GLContext' objects>, '__weakref__': <attribute '__weakref__' of 'GLContext' objects>})

__module__ = 'fsleyes.gl'

__weakref__

list of weak references to the object (if defined)

class fsleyes.gl.OffScreenCanvasTarget(width, height)

Bases: object

Base class for canvas objects which support off-screen rendering.

__init__(width, height)

Create an OffScreenCanvasTarget. A RenderTexture is created, to be used as the rendering target.

Parameters

• width – Width in pixels

• height – Height in pixels

_setGLContext()

Configures the GL context to render to this canvas.

_draw(*a)

Must be provided by subclasses.

getAnnotations()

Must be provided by subclasses.

GetSize()

Returns a tuple containing the canvas width and height.

GetScaledSize()

Returns a tuple containing the canvas width and height.

Refresh(*a)

Does nothing. This canvas is for static (i.e. unchanging) rendering.

FreezeDraw()

Does nothing. This canvas is for static (i.e. unchanging) rendering.

ThawDraw()

Does nothing. This canvas is for static (i.e. unchanging) rendering.

FreezeSwapBuffers()

Does nothing. This canvas is for static (i.e. unchanging) rendering.

ThawSwapBuffers()

Does nothing. This canvas is for static (i.e. unchanging) rendering.

EnableHighDPI()

Does nothing. This canvas is for static (i.e. unchanging) rendering.

draw()

Calls the _draw() method, which must be provided by subclasses.

getBitmap()

Return a (height*width*4) shaped numpy array containing the rendered scene as an RGBA bitmap. The bitmap will be full of zeros if the scene has not been drawn (via a call to draw()).

saveToFile(filename)

Saves the contents of this canvas as an image, to the specified file.

__dict__ = mappingproxy({'__module__': 'fsleyes.gl', '__doc__': 'Base class for canvas objects which support off-screen rendering. ', '__init__': <function OffScreenCanvasTarget.__init__>, '_setGLContext': <function OffScreenCanvasTarget._setGLContext>, '_draw': <function OffScreenCanvasTarget._draw>, 'getAnnotations': <function OffScreenCanvasTarget.getAnnotations>, 'GetSize': <function OffScreenCanvasTarget.GetSize>, 'GetScaledSize': <function OffScreenCanvasTarget.GetScaledSize>, 'Refresh': <function OffScreenCanvasTarget.Refresh>, 'FreezeDraw': <function OffScreenCanvasTarget.FreezeDraw>, 'ThawDraw': <function OffScreenCanvasTarget.ThawDraw>, 'FreezeSwapBuffers': <function OffScreenCanvasTarget.FreezeSwapBuffers>, 'ThawSwapBuffers': <function OffScreenCanvasTarget.ThawSwapBuffers>, 'EnableHighDPI': <function OffScreenCanvasTarget.EnableHighDPI>, 'draw': <function OffScreenCanvasTarget.draw>, 'getBitmap': <function OffScreenCanvasTarget.getBitmap>, 'saveToFile': <function OffScreenCanvasTarget.saveToFile>, '__dict__': <attribute '__dict__' of 'OffScreenCanvasTarget' objects>, '__weakref__': <attribute '__weakref__' of 'OffScreenCanvasTarget' objects>})

__module__ = 'fsleyes.gl'

__weakref__

list of weak references to the object (if defined)

fsleyes.gl.WXGLMetaClass

alias of builtins.type

class fsleyes.gl.WXGLCanvasTarget

Bases: object

Base class for wx.glcanvas.GLCanvas objects.

It is assumed that subclasses of this base class are also subclasses of wx.glcanvas.GLCanvas. Sub-classes must override the following methods:

_initGL	This method should perform any OpenGL data initialisation required for rendering.
_draw	This method should implement the OpenGL drawing logic - it must be implemented by subclasses.

static canToggleHighDPI()

Return True if high-DPI scaling can be toggled, False otherwise.

Under GTK, high DPI support is not possible with wxPython < 4.0.7, as wx.Window.GetContentScaleFactor always returns 1. Under GTK and from wxPython 4.0.7 onwards, GL canvases are scaled automatically.

Under macOS and wxpython < 4.1.0, high DPI must be explicitly requested for GL canvases via a Cocoa API call. This can be done via the EnableHighDPI() method.

Under macOS and with wxpython >= 4.1.0, GL canvases are scaled automatically.

__init__()

Create a WXGLCanvasTarget.

_initGL()

This method should perform any OpenGL data initialisation required for rendering. Must be implemented by subclasses.

getAnnotations()

Must be provided by subclasses.

_draw(*a)

This method should implement the OpenGL drawing logic - it must be implemented by subclasses.

Note

When runing with an on-screen display, this method should never be called directly - call the Refresh() method instead.

_setGLContext()

Configures the GL context for drawing to this canvas.

This method should be called before any OpenGL operations related to this canvas take place (e.g. texture/data creation, drawing, etc).

GetSize()

Returns the current canvas size.

GetContentScaleFactor()

Overrides wx.Window.GetContentScaleFactor.

Calls the base class implementation, except where wxpython 3.0.2.0 is being used, as the method does not exist in that version. In this case, 1.0 is returned.

GetScale()

Returns the current DPI scaling factor.

GetScaledSize()

Returns the current canvas size, scaled by the current DPI scaling factor.

Refresh(*a)

Triggers a redraw via the _draw() method.

FreezeDraw()

Freezes updates to the canvas. See ThawDraw().

ThawDraw()

Unfreezes canvas updates. See FreezeDraw().

_WXGLCanvasTarget__onEraseBackground(ev)

Called on wx.EVT_ERASE_BACKGROUND events. Does nothing.

_WXGLCanvasTarget__onPaint(ev)

Called on wx.EVT_PAINT events. Schedules Refresh() to be called on the idle loop.

_WXGLCanvasTarget__realDraw(*a)

Called when the canvas needs to be refreshed.

This method calls _initGL() if it has not already been called. Otherwise, it calls the subclass _draw() method.

__dict__ = mappingproxy({'__module__': 'fsleyes.gl', '__doc__': 'Base class for :class:`wx.glcanvas.GLCanvas` objects.\n\n It is assumed that subclasses of this base class are also subclasses of\n :class:`wx.glcanvas.GLCanvas`. Sub-classes must override the following\n methods:\n\n .. autosummary::\n :nosignatures:\n\n _initGL\n _draw\n ', 'canToggleHighDPI': <staticmethod object>, '__init__': <function WXGLCanvasTarget.__init__>, '_WXGLCanvasTarget__onEraseBackground': <function WXGLCanvasTarget.__onEraseBackground>, '_WXGLCanvasTarget__onPaint': <function WXGLCanvasTarget.__onPaint>, '_initGL': <function WXGLCanvasTarget._initGL>, 'getAnnotations': <function WXGLCanvasTarget.getAnnotations>, '_draw': <function WXGLCanvasTarget._draw>, '_WXGLCanvasTarget__realDraw': <function WXGLCanvasTarget.__realDraw>, '_setGLContext': <function WXGLCanvasTarget._setGLContext>, 'GetSize': <function WXGLCanvasTarget.GetSize>, 'GetContentScaleFactor': <function WXGLCanvasTarget.GetContentScaleFactor>, 'GetScale': <function WXGLCanvasTarget.GetScale>, 'GetScaledSize': <function WXGLCanvasTarget.GetScaledSize>, 'Refresh': <function WXGLCanvasTarget.Refresh>, 'FreezeDraw': <function WXGLCanvasTarget.FreezeDraw>, 'ThawDraw': <function WXGLCanvasTarget.ThawDraw>, 'FreezeSwapBuffers': <function WXGLCanvasTarget.FreezeSwapBuffers>, 'ThawSwapBuffers': <function WXGLCanvasTarget.ThawSwapBuffers>, 'SwapBuffers': <function WXGLCanvasTarget.SwapBuffers>, 'EnableHighDPI': <function WXGLCanvasTarget.EnableHighDPI>, 'getBitmap': <function WXGLCanvasTarget.getBitmap>, '__dict__': <attribute '__dict__' of 'WXGLCanvasTarget' objects>, '__weakref__': <attribute '__weakref__' of 'WXGLCanvasTarget' objects>})

__module__ = 'fsleyes.gl'

__weakref__

list of weak references to the object (if defined)

FreezeSwapBuffers()

Freezes canvas fron/back buffer swaps, but not canvas drawing. See ThawSwapBuffers().

ThawSwapBuffers()

Unfreezes canvas fron/back buffer swaps. See FreezeSwapBuffers().

SwapBuffers()

Overrides wx.GLCanvas.SwapBuffers. Calls that method, but only if FreezeSwapBuffers is not active.

EnableHighDPI(enable=True)

Attempts to enable/disable high-resolution rendering.

getBitmap()

Return a (width*height*4) shaped numpy array containing the rendered scene as an RGBA bitmap.