IFP - An Interactive Fiction Plugin Scheme

Author: Simon Baldwin <simon_baldwin@yahoo.com>
Date: 1st March 2003

Contents

Summary
Overview and Introduction
Quick Start Guide
IFP Goals
IFP Architecture
IFP Implementation
Ambitious IFP Features
More Details on IFP Functions
Notes on Building IFP Source
Notes on Running Games
Bugs and Omissions
Unfinished Business
Acknowledgments

Appendix A - Strdup, Optimization, and IFP
Appendix B - X Resources for a More Attractive Xglk

IFP Copyright

Summary

IFP plays several different types of Interactive Fiction game files using a single game playing application program, and a set of game interpreter plugins. The program automatically determines how to play the game from the data in the game file, and the different game files are all handled seamlessly. The current implementation contains plugins to handle game files in the following formats:

Z-machine, TADS, HUGO, Alan, Glulx, AGT, AdvSys, Level 9, Magnetic Scrolls, and Blorb

For Blorb files, IFP recognizes ZCOD (Z-machine) and GLUL (Glulx) executable types.

IFP can directly handle Compressed, Gzipped, or Bzipped game files, and game files, compressed or otherwise, contained in Zip, Tar, or Cpio archives. It can also handle URL references to game files, compressed game files, or game archives.

Overview and Introduction

I've been playing Interactive Fiction games for a while, using a dazzling array of IF interpreters for the various different game file formats in existence -- Frotz, Xzip, Kwest and so on for Inform games, Tadsr for TADS games, He for HUGO games, and so on. Some of these interpreters I use are graphical, and some not. Each is an individual binary. Some of them I have had to build from source, occasionally needing to fix some 'C' construct or another in the source to get it past today's Gcc compiler.

I realized that what I really wanted to have was just one interpreter binary that would handle all of these games formats, and let me put an icon on my KDE desktop onto which I could drag and drop .z5 Inform files, .hex HUGO files, .gam TADS files, and so on. When handed the game data file, this single interpreter would work out what type of game it was, then run it using a common look-and-feel.

Along the way, I also started thinking about modularization of IF interpreter engines, and wondered if there was some way to make the code in them more reusable. Media players such as Xmms, and the Windows Media Player, use plugins to decode content, and are in themselves just methods to deliver that decoded content. The same sorts of ideas should also be applicable to IF interpreters -- create one or more main applications that know nothing about IF game file formats, but do know how to load and run IF interpreter engine plugins, then create one (or more) IF interpreter engines plugins for each game file format.

The result is IFP. Depending on your viewpoint, it is either an extension to, or an adjunct to, Andrew Plotkin's excellent Glk API. Using IFP, and suitably equipped with plugins, an application given a game data file can find, load, and execute a plugin to run the game contained in that file. Also depending on your viewpoint, IFP might be useful, or not; either way, I have something I can use with multiple game formats.

IFP is not a standard, a proposal, or specification. It is an architecture and an implementation. There's real code in it. Build the code, and play multiple game file formats through one application program. As an added bonus, you can, if you wish, hand that application a gzipped, compressed, packed, or bzipped game file, a tar, cpio, or zip archive, or even an FTP or HTTP URL; the program will still find, and run, the game contained in the compressed data, archive, or URL data stream.

Quick Start Guide

The current version of IFP is for Linux only, and offers both X and curses (character) game-playing programs.

The fastest route to running IFP is to install it in binary form. If your system understands RPM packages, you can install IFP from the file

ifp-1.1-1.i386.rpm

The matching source code for this binary RPM is in the file ifp-1.1-1.src.rpm.

Alternatively, if your system understands Debian packages, you can install IFP from the file

ifp_1.1-2_i386.deb

If your Linux is neither RedHat nor Debian based, and you don't want to be bothered building source, you can extract just the IFP binaries from the Debian binary package with the following commands:

ar x ifp_1.1-2_i386.deb data.tar.gz
mv data.tar.gz ifp-1.1-binaries.tgz

The IFP binary packages contain everything needed to run games, and also includes a very small selection of games. If you don't have any IF game files to hand, the source IFP distributions contain several you can use to get started. Or you can download a selection from the IF Archive web site.

IFP installs its main game interpreters in the paths

/usr/bin/ifpe
/usr/bin/xifpe
/usr/bin/xlegion

It also installs a very small selection of interactive fiction games, really just a taster, for convenience, in the directory

/usr/lib/games/ifp

For a huge selection of games, all available for free download, please visit the IF Archive at http://www.ifarchive.org.

Xifpe is IFP's baseline IF game playing program. There's a curses version, too, called ifpe. You can also use xlegion. It's a slightly fuller featured IF game playing program (with a less utilitarian name).

To run a game, give it as an argument to one of these interpreter binaries, for example

xifpe /usr/lib/games/ifp/weather.z5

This will probably warn you that it is using a default value for IF_PLUGIN_PATH, but the game should run, providing its type is one for which IFP has a plugin. IF_PLUGIN_PATH is the path that IFP uses to locate interpreter plugins. Unless you are experimenting with plugins, you should probably just set this variable in your .profile or other configuration file to have a permanent value of

/usr/lib/ifp:/usr/local/lib/ifp

and leave it at that.

IFP can run games using compressed archives and URLs directly, so some other things you might want to try are

xifpe /home/my_account/my_files/bugged.zip
xlegion http://www.ifarchive.org/if-archive/games/zcode/curses.z5

Building IFP from source code

To build IFP from source code, download a copy of the file

ifp-1.1.tgz

or extract this file from the source RPM distribution with rpm2cpio. Unpack the archive into a suitable directory:

gunzip -c <ifp-1.1.tgz | tar xvf -

Build the current version of IFP, with

cd ifp-1.1
make all

If all goes well with the build, try running a game. There are samples included with the IFP distribution, so if you don't have any IF games to hand that you'd like to try, try the command

IF_PLUGIN_PATH="ifp:plugins" ifp/xifpe games/weather.z5

This sets a value for the variable IF_PLUGIN_PATH, used by the IFP libraries when deciding where to look for IFP plugins to load, then runs xifpe on the example game weather.z5. Provided the IFP build succeeded, you should see a Glk frame appear on your X display, with the game running in it.

You can now experiment with the other games in the 'games' subdirectory, and also with the games in the 'derivatives' subdirectory (these are the same games, but in assorted compressed formats). Some other things to try are

IF_PLUGIN_PATH="ifp:plugins" \
ifp/xifpe derivatives/bugged.zip
IF_PLUGIN_PATH="ifp:plugins" \
ifp/xlegion http://www.ifarchive.org/if-archive/games/zcode/curses.z5

If you're going to do much of this, you might want to set a permanent value for IF_PLUGIN_PATH, and to install IFP in /usr/local on your system.

For game formats that distribute their games in multiple files, there is usually one file that IFP will use to identify the correct interpreter for a game. For example, the Alan game in bugged.zip contains the two files

bugged.acd
bugged.dat

In this case, the file with the extension '.acd' is the one that IFP recognizes as an Alan game.

For AGT games, the identification file is the one with the extension '.d$$'. If you enter this file on the command line, remember to escape the '$' characters. AGT games use filenames which are extremely awkward for UNIX systems. For this reason, if no other, it's handy to play games using IFP's ability to run games directly from inside zip files.

If you find something that doesn't seem to be working right, please first check the Bugs and Omissions section below to find out if what you're seeing is known, new, or maybe just a feature. I'll happily accept bug reports on IFP, but if you do discover something that fails, please try to send me the exact details of how to reproduce the fault. Or best of all, of course, you can send me the fix!

IFP Goals

The goals of the IFP scheme are:

Use existing Glk-enabled interpreter code as much as possible.

IFP needs to be able to run with interpreter code that has existed for a while, and has therefore become stable. There is no gain in having to write new interpreter engine plugins from scratch just for IFP.

Make as few modifications to the code of the existing interpreters as is possible.

The process of turning an existing Glk interpreter into an IFP interpreter plugin must be a very simple one, otherwise nobody is going to bother to do it.

Provide an extremely simple, and high-level, API.

All of the difficulty of loading and managing multiple IF plugin engines must be handle by the IFP implementation, and an IFP user program should be able to accomplish relatively complex plugin tasks using a simple, predictable, and well-organized API.

IFP Architecture

The main IF interpreters (Tadsr, Glulxe, Nitfol, and so on) are all Linux "main" programs, in the sense that they contain the symbol "main", to which the Linux Glibc transfers control on process startup.

Actually, when they become Glk programs, their "main" comes from the Glk libraries, and they implement glkunix_startup_code() and glk_main(), to which the Glk "main" links. But, they remain "main"-ish in their nature -- they take an argument count and some string arguments, the last of which is always the game file to open and run, and they run until completion, either by returning from glk_main() (the Glk analog to "main"), or by calling glk_exit() (the Glk analog to "exit(1)", and quite why Glk always exits a program with a status code of 1 is a mystery, since 0 would be much more normal).

Hello, world as a shared object

Here's the traditional simplest possible Linux (and UNIX) program:

$ cat hw.c
#include <stdio.h>
int main () {
printf ("Hello, world\n");
}

Compile and build into a simple program with the following:

$ cc -o hw hw.c
$ ./hw
Hello, world

Nothing very revolutionary so far. But with a few changes to the build process, we can turn this into a shared object (.so) instead of a simple executable binary:

$ cc -c hw.c
$ ld -shared -o hw.so hw.o

To run the code in our new shared object takes a little judicious use of the dynamic linker library:

$ cat loadhw.c
#include <stdio.h>
#include <dlfcn.h>
int main () {
        void    *handle;
        int     (*so_main) ();
        handle = dlopen ("./hw.so", RTLD_NOW);
        if (handle == NULL) {
                fprintf (stderr, "%s", dlerror ());
                exit (1);
        }
        so_main = dlsym (handle, "main");
        if (so_main == NULL) {
                fprintf (stderr, "%s", dlerror ());
                exit (1);
        }
        printf ("Calling shared object main...\n");
        (*so_main) ();
        printf ("...back from shared object main\n");
        dlclose (handle);
        return 0;
}
$ cc -o loadhw loadhw.c -ldl
B4linux:/tmp$ ./loadhw
Calling shared object main...
Hello, world
...back from shared object main

These are the beginnings of a plugin scheme. What used to be a main program can be turned into a shared object, and that shared object can be loaded and run from a new main program. More importantly, this requires no source code changes to the original main program.

Note that hw.c doesn't call exit() at the end of main(). If it did, we would not get to see the message "...back from shared object main", so if we cared about this we'd need to make some refinements. For simplicity, the example also doesn't pass in arguments to main(). But this is the basic idea behind IFP plugins.

In a little more detail, here is one view we could take of an existing Glk interpreter, say Tadsr:

Xglk library

------------------->
Glk control

<-------------------
Glk services

TADS interpreter code

The Glk control interface consists of calls to glkunix_startup_code(), and glk_main(), and the Glk library makes those calls into the TADS interpreter code from its own, very real, main(). In return, the TADS interpreter makes Glk service calls to request open files, windows, and so on. Tadsr, built normally, exists on disk as on single executable file, containing both the TADS interpreter code and the Xglk library.

Using some linker options and the dynamic library API, we can turn this into two system files. One is a shared object containing the TADS interpreter code, and the other is a main executable file, containing the Xglk library, and the code to load the TADS interpreter. Using the dynamic linker library, we can glue these two system files together into a single runnable application.

Extend this to other existing Glk interpreters to arrive at the idea of a plugin -- the main executable program loads all the plugins it can find, and sees which of them can run the game file it has been given.

Keeping symbol namespaces separate

Depending on what linker options are used to build the shared object and the main program, the symbols in one may, or may not, be visible to the other. It is also possible to make symbols from a shared object visible to other shared objects loaded by the main program.

If we want to be able to load multiple interpreter plugins into our main program, we'd probably prefer that they can't all see each other's symbols, otherwise we're likely to run into some symbol name clashes; for example, all interpreters define, and export, glkunix_startup_code() and glk_main().

And to be safe, let's say that interpreter plugins don't get to see the symbols in our main program either; we probably define glkunix_startup_code() and glk_main(), too. This means that we have to tell the plugin the addresses of each Glk function that it wants to call, so the the Glk services interface works. But it does mean that we can control precisely what a plugin can and cannot do with our main program, and it also means that the risk of symbols accidentally colliding is much smaller.

Recognizing game file data

To make the plugin scheme work, we want a way of being able to tell if an interpreter plugin can run a particular game file. It is tempting to simply call the plugin's glkunix_startup_code() and see if that succeeds. Unfortunately, most current Glk interpreters simply store the arguments (including the game filename) and return TRUE, and then wait until a call to glk_main() to actually read and load the game. By this stage, our main program will already be committed to using that plugin (there's no way from within glk_main() to return an error status), so this technique won't work.

Because of this, IFP uses a pattern matching scheme, vaguely similar to that in file(1), to determine whether an engine can run a game. As part of the process of "pluginifying" an existing interpreter, it's necessary to define a pattern to look for in a game file, and to say where to look for it. On loading the interpreter, the main program reads that pattern and checks it against the game data. If it matches, then the main program calls the plugin's glkunix_startup_code() function. Assuming that succeeds, and most do regardless of what they are passed, the main program is then committed to using that plugin.

A possible pitfall to this is false recognition. A file might look like an Inform game, but actually contain a JPEG image. Or the game file might simply be corrupted. Some games are easy to recognize, for example TADS, which has the header "TADS" at the top of the game file, or Glulx, which starts "Glul". Some are horribly difficult, like Inform, which has the game version number of 0x03 to 0x08 as the first byte in the file, and very little else from which to discern the file type in it. See the /usr/share/misc/magic (or equivalent) file on a Linux system for details about why Inform game data files are hard to identify.

In practice, however, simple pattern recognition seems to work reasonably well. What we need to do is differentiate between different game file types, and not necessarily between all possible file types. It's not disastrous if we misidentify a JPEG image as an Z-machine file, since (a) the Z-machine plugin will almost certainly refuse it, and (b) no other interpreter plugin could have used the file anyway.

IFP Implementation

When built, IFP consists of two libraries:

libifp.a
libifppi.a

along with two baseline interpreter executables called ifpe and xifpe, and a slightly beefier executable called xlegion. All executables are built on top of the IFP and Glk libraries.

Libifp is the "client-side" library, the one to link to if writing a main application to use IF plugins.

Libifppi is the "plugin-side" library. This is the one to link into the plugins themselves.

Between them, these two libraries co-operate in terms of passing interfaces and other data. They take care of the mechanics of managing the plugin scheme, so that the main application doesn't need to worry about too many details, and so that "pluginifying" an existing Glk interpreter is straightforward. The version of libifppi built into a plugin must match the version of libifp in the main program, otherwise IFP will refuse to load the plugin.

Converting an existing interpreter into a plugin

To convert an interpreter to a plugin requires two simple steps: writing the 'C' module that defines the header information for the plugin, and altering the interpreter's Makefile to build the plugin shared object.

Every plugin contains a small 'C' module which provides the IFP loader with some information about the plugin. The most important piece of information in here is going to be the detail about the type or types of files the plugin handles. There are a few other items, too. The 'C' module needs to export one symbol, ifpi_header, and doesn't need to have any executable code in it at all. Here is the definition of the ifp_header structure, used to create ifpi_header:

struct ifp_header {
        int     ifp_version;            /* Version of IFP the engine uses. */
        char    *build_timestamp;       /* Date and time of plugin build */
        char    *engine_type;           /* Data type the engine runs. */
        char    *engine_name;           /* Unique engine name. */
        char    *engine_version;        /* Engine's version number. */
        char    *blorb_pattern;         /* If Blorb-capable, the exec type. */
        int     acceptor_offset;        /* Offset in file to "magic". */
        int     acceptor_length;        /* Length of "magic". */
        char    *acceptor_pattern;      /* Regular expression for "magic". */
        char    *author_name;           /* Interpreter author's name. */
        char    *author_email;          /* Interpreter author's email. */
        char    *engine_home_url;       /* Any URL regarding the interpreter. */
        char    *builder_name;          /* Engine porter/builder's name. */
        char    *builder_email;         /* Engine porter/builder's email. */
        char    *engine_description;    /* Miscellaneous engine information. */
        char    *engine_copyright;      /* Engine's copyright information. */
};

The ifp_version is used to check that the plugin's libifppi and the main libifp match. There's a macro defined for it. The build_timestamp is free-format, and __DATE__", "__TIME__ is a good choice, but it can be blank or NULL.

The engine_type, name, and version, are all free-format, but note that IFP considers two plugins to be identical if their name and version strings match, and will not load duplicate plugins.

Skipping ahead for a moment, the author details, engine URL, builder details, miscellaneous information, and copyright are all free-format, and can be left NULL if desired. All of these fields are for information only. A main program might display them when using the plugin, but IFP does nothing with them internally.

Acceptor pattern matching

A plugin can elect to accept "native" game data files, Blorb-encapsulated data files, or both. It can also elect to accept neither, but that isn't very useful.

For "native" game data files, the blorb_pattern is ignored. The acceptor_offset, length, and pattern contain the critical information about how to spot a file that this plugin engine can handle. Offset is the position in the file where data identifying the file's type might be found. It's normally at or very close to the start of the file. Length indicates the length of the identifying pattern.

The acceptor_pattern is a regular expression to apply to an "asciified" version of the data found at offset/length. This involves turning the data into a string composed of hexadecimal characters. Unfortunately, regular expression matches don't apply nicely to binary data, so the data needs to be converted to ASCII before any pattern match. This is awkward, since it makes otherwise simple cases look obscure. But it does make otherwise impossible cases possible.

Here's an example. For TADS, the file identifier is "TADS2 bin". The offset is 0, and length is 9. After "asciification" of the first nine bytes of a TADS data file, we get the string "54 41 44 53 32 20 62 69 6e". So the pattern to set here is "^54 41 44 53 32 20 62 69 6e$". Any data file that matches this is likely to be TADS data.

For Inform, the offset is 0, the length 2, and the acceptor pattern is "^0[3-8] 0[0-F]$". Kind of vague, isn't it? But that's Inform for you -- the Z-machine file format goes back a long way.

In the case of a Blorb file, the acceptor_offset, length, and pattern are ignored, and everything IFP needs to know about the plugin is found in the blorb_pattern field.

For Blorb files, IFP finds the first Blorb "Exec" resource in the file, and determines its type. At present, two types are likely -- 'ZCOD' and 'GLUL'. The type found is first converted into a four byte string (Blorb normally encodes types as four bytes packed into a one 32-bit word), the string is then "asciified", and finally, the "asciified" version is matched against the regular expression blorb_pattern that the plugin advertises.

For example, Blorb files containing Glulx games will have the 'GLUL' as the Exec resource type in the Blorb file. IFP converts the 'GLUL' first into a four byte string "GLUL", then "asciifies" the string to obtain "47 6C 75 6C". So the Blorb acceptor for 'GLUL' Blorb files is "^47 6C 75 6C$".

While necessary for recognizing "native" game file data, "asciifying" Blorb types is not strictly necessary. It is however easier to think in terms of one form of pattern match rather than two, so IFP does matches this way for symmetry.

Getting the acceptor patterns right, or near enough, is the hardest part of converting to a plugin. The following alias for the hexdump command might help in dealing with actual games data files if you're not familiar with the binary format:

alias hd='hexdump -e '\''"%06.6_ax " 16/1 "%02x "'\'' -e '\''" " 16/1 "%_p" "\n"'\'''

Who says Linux is cryptic? This gives a nice little hexadecimal dump output, combined with ASCII characters to the right.

A tool for building an IFP header 'C' module

Though it's not difficult to write the 'C' module that sets up a plugin's ifpi_header, it is a little awkward, since there is a mix of IFP administrative information and plugin-specific information in here.

To make it a bit easier, IFP contains a small tool called ifphdr. This utility takes a set of definitions in shell script like format (actually, it is shell script format -- the definitions are just 'sourced' into ifphdr), checks the mandatory ones, defaults any absent ones to NULL, and outputs the 'C' code that can be compiled to create the ifpi_header.

So, again using the example of TADS, here is the complete definition file for the TADS plugin data:

IFP_ENGINE_TYPE="TADS"
IFP_ENGINE_NAME="Tadsr"
IFP_ENGINE_VERSION="2.5.7"
IFP_ACCEPTOR_OFFSET=0
IFP_ACCEPTOR_LENGTH=9
IFP_ACCEPTOR_PATTERN="^54 41 44 53 32 20 62 69 6e$"
IFP_AUTHOR_NAME="Michael J. Roberts"
IFP_AUTHOR_EMAIL="mjr_@hotmail.com"
IFP_BUILDER_NAME="Simon Baldwin"
IFP_BUILDER_EMAIL="simon_baldwin@yahoo.com"
IFP_ENGINE_DESCRIPTION=\
"This is a Unix version of the TADS 2.5.7 source distribution, provided to
make it easier for Unix users to compile the distribution on their own
systems.\n"
IFP_ENGINE_COPYRIGHT="Copyright (c) 2002 by Michael J. Roberts.
All Rights Reserved.\n"

To build the plugin header using ifphdr requires something as short as:

ifphdr myplugin.hdr myplugin_header.c
cc -I../xglk -c myplugin_header.c

Plugin Makefile changes

The other step to converting an interpreter into a plugin is to alter the Makefile so that it builds a shared object, and links in libifppi. This should be straightforward, and is normally something like this:

PLIBS = -L ../../ifp -lifppi
POBJS = ../../myplugin_header.o
myplugin-0.0.1.so: $(OBJECTS) $(POBJS)
$(LD) -u ifpi_force_link -shared -Bsymbolic
-o $@ $(OBJECTS) $(GLKOBJS) $(PLIBS) $(POBJS) -lc

This defines myplugin-0.0.1.so as the shared object to build. $(OBJECTS) is the exact same set of object files that get built into the standalone interpreter. $(POBJS) is a reference to the module that contains the ifpi_header definition we wrote or created with ifphdr above. And we link in libifppi as well.

Note that we don't link in any Glk library. We don't need to; in fact, we don't want to. Libifppi contains symbols that resolve all of the glk_ function calls that the interpreter code makes. It doesn't, of course, contain the functions. But what it does know is how to get those function calls back to the main program that will load it.

The -u ifpi_force_link ensures that all the necessary bits of libifppi get included in myplugin-0.0.1.so. Without it, the linker may consider pieces of libifppi to be irrelevant, since no interpreter code necessarily calls them, and they do need to be present.

Once these two simple steps are completed, the interpreter is ready for use by IFP.

IFP functions for using plugins

Libifp, the IFP client-side library, contains quite a few API calls, but many are for specialized operations that aren't usually necessary. IFP API calls all begin with "ifp_", and the following somewhat stylized code example shows all that is required to find and run a plugin for a data file:

ifp_pluginref_t    plugin;
plugin = ifp_manager_locate_plugin (data_file_path);
if (plugin != NULL)
    {
        ifp_manager_run_plugin (plugin);
        ifp_loader_forget_plugin (plugin);
    }
else
    {
        printf ("No plugin engine found for the data file\n");
    }

Not much to it, is there? The heart of the operation is ifp_manager_locate_plugin(). This takes a data file, then searches the plugin load path for each available IF plugin. If any matches the data in the data file, it return the plugin's address. This is an opaque "handle" to a plugin, and its type is ifp_pluginref_t. The interpreter's glkunix_startup_code() function has already been called, and the interpreter should be ready to go. If it returns NULL, then the IFP loader found no suitable plugin interpreter for the data in the file path passed to it.

Assuming IFP found a plugin, the function ifp_manager_run_plugin() runs the interpreter.

Behind the scenes

Underneath ifp_manager_locate_plugin(), quite a bit of work takes place. Firstly, the IFP library searches for all available IF plugins using the path set in the environment variable IF_PLUGIN_PATH. Each plugin is then loaded, and examined to see if the acceptor in its ifpi_header symbol matches the data file.

Where there is a data match, the IFP library hands the plugin an interface structure which tells it the addresses of each Glk library function. This is how the plugin transfers Glk calls out through libifppi. Assuming the plugin accepts the Glk interface, and its glkunix_startup_code() function returns successfully when called, it becomes the Chosen One, and ifp_manager_locate_plugin() returns a reference to it.

All of this negotiation about acceptor patterns and interfaces happens between the two libraries libifp and libifppi. There is no need for any client code to worry about it at all, unless it requires finer control.

The function ifp_manager_run_plugin() is much simpler -- it is primarily a wrapper round the plugin's glk_main() function, although it does some housekeeping for IFP, and also ensures that only one plugin can actually be active at a time.

Overriding glkunix_startup_code and glk_main in a plugin

A plugin might need to do extra stuff in its startup and main code that doesn't usually happen in a normal build. If this is the case, the plugin can define and export the functions ifpi_glkunix_startup_code and ifpi_glk_main. The IFP loader looks for these symbols first, then reverts to glkunix_startup_code and glk_main if they don't exist.

In normal circumstances, however, the usual glkunix_startup_code and glk_main are fine, and the only additional symbol needed for a plugin form of an interpreter is ifpi_header.

Including Glk in the main program

The main program that contains this IFP client code needs to be linked with a Glk library, even if it never makes a single Glk library call itself. This is because it sends the function addresses of all Glk functions to the plugins it loads, so that their interpreters can make Glk calls when they need to. The mechanics of how these function addresses move between the main program and the plugins are all handled by Libifp and Libifppi, but a Glk library is the one major non-IFP code dependency with IFP.

Ambitious IFP Features

As I've described it above, the IFP scheme, and implementation, are enough to create an IF games interpreter that will load and run the correct plugin for a particular game file on disk. However, IFP also contains several additional, and ambitious, features designed to generally enhance usability.

The state of implementation of these features varies. Each is present in some form or another, but may be less than complete. They're highly likely to change with any future versions of IFP, too.

Repeating games

When a game interpreter calls glk_exit(), it has finished running. It is expecting that the entire process will stop there, since it has finished (so what else could remain to be done?). However, for IFP, this isn't entirely satisfactory. It is useful if IFP can, when one game plugin finishes, go off and run another, or maybe run the same game again. In general, it's undesirable in a GUI for the application to shut down until the user shuts it down.

IFP handles this by redirecting calls the interpreter makes to glk_exit() (remember that we pass it an interface structure containing the addresses of all Glk calls, so IFP gets to see every Glk call that an interpreter makes). When the interpreter calls glk_exit(), IFP sets things up so that one of its functions gets the call instead. Using the miracle of the 'C' setjmp/longjmp calls, IFP handles this event as if the interpreter's glk_main() had simply returned.

IFP also redirects any calls the interpreter might make to exit() (though of course, none should if they adhere to Glk API guidelines). A call to exit() is treated as a call to glk_exit(). (If an interpreter really wants to stop the process, it can call _exit()).

Now, an interpreter won't "accidentally" stop the process. But there's still the matter of "garbage collection" to consider.

Because interpreters are expecting glk_exit() to terminate the process, most will happily call it while still holding system resources. Chief among these is malloc()'ed memory. A "pluginified" interpreter will often not free its heap memory before exiting.

To deal with this, IFP intercepts all calls that the interpreter code makes to malloc() and its close relatives. It does this by simply defining the functions malloc(), free(), and so on, in libifppi. Since these symbols are "weak" in libc, there's no problem in overriding them.

Each intercepted malloc() and similar call from an interpreter is passed back to the main application. This calls the real malloc() or appropriate function, then updates an internal table with information about what memory addresses the interpreter has malloc()'ed but not yet freed. When the interpreter finishes, IFP scans this table, and automatically free()'s any memory that the interpreter left unfreed when it called glk_exit() (or just returned from glk_main()).

IFP also uses this same technique to try to reduce file descriptor leaks that might also be caused by an interpreter opening but not closing a file. When it comes to cleaning up Glk, IFP has a much easier time. Glk is very helpful in that it offers ways to query and destroy most of the resources it has accumulated while running a game.

At present, the baseline IFP interpreter, xifpe, and its curses cousin, ifpe, do not try to repeat games or start new games when an interpreter plugin completes running; only xlegion currently uses this feature.

Chaining plugins

Since IFP contains two parts, a client-side library that knows how to find and load plugins, and a plugin-side library that knows how to be loaded by the client, it's interesting to stick these together into one single plugin.

A chaining plugin is a plugin that takes on a portion of the game interpreting task, then passes off the real work to some other plugin. There are three included in IFP at present:

Unzip

This plugin handles gzipped, bzipped, compressed, and packed data.

Pkunzip

This plugin handles data zipped with the (pk)zip utility.

Tar

This plugin handles tar and cpio archives

What makes these interesting is that they do not interpret the game. What they do is to uncompress or unarchive the file that they are handed into files in /tmp, then use their built-in IFP loader functions to search for another plugin that is able to handle the data they just uncompressed, acting like an middleman in the whole affair. In this way, a file such as weather.z5.gz, or an Alan file in its "native" format of bugged.zip can be handled directly by IFP.

It's also possible for a chaining plugin to load and run another copy of itself. This means that uncompressions are not limited to one level -- a file such as weather.z5.gz.gz.gz could be interpreted (though it might be a little slow drilling down through multiple uncompressions).

There are quite a few special tricks, some of them rather ugly, in IFP to permit chaining plugins to work this way. But they do work.

Using URLs

IFP contains functions to allow it to directly handle URLs. At present, file:, http:, and ftp: URLs are supported. The IF Archive offers all of its games through both an HTTP server and an FTP service.

URL resolution is relatively simple. IFP contains small, built-in HTTP and FTP clients. On being handed a URL that it needs to use, IFP uses the relevant client to download the data from the URL and store it on a disk temporary file. That file can then be passed to the client-side loader functions, to find the right plugin for the downloaded data, and the whole business of loading and running the right plugin continues as normal from there.

Having URLs included in IFP allows the main game program to run games from the IF Archive directly, for example:

xifpe http://www.ifarchive.org/if-archive/games/zcode/curses.z5

URLs go to some extremes to try to ensure that temporary files are deleted from /tmp when the interpreter program exits.

URLs can be resolved either synchronously or asynchronously. Asynchronously resolving a URL will allow the URL resolve function to return as soon as it has negotiated and begun to download from the server. A program using this form of download needs to poll the URL to see if the download has completed, and will not be able to access URL data until it has. It can, however, go off and do other things, and check back on the progress of the download later.

Interpreter options

IFP doesn't have the luxury of being able to ignore the topic of setting interpreter options. The method it uses is to allow the main program to pre-register a set of options to be passed as part of the argc/argv array to glkunix_startup_code() whenever a plugin of a particular type is loaded and run.

At some point, I hope that this mechanism can be connected to something in the main program that will offer the user a window with which to configure options for each plugin available on the system. Such a facility should be able to use the Glk arguments array to discover what options a plugin takes, offer them to the user, and then pre-register each with IFP.

For now, this part of IFP is under active development, and is used only to turn off the highly annoying warnings emitted almost continuously by the Nitfol interpreter.

Legion

Legion is a work-in-progress to produce an IF interpreter that will cycle through games, print out details of the selected plugins, and generally take advantage of, and test, some of the extra features above that IFP tries to implement. It's not complete, in fact it's barely started. However, it builds and runs, and is there if you wish to play with it.

More Detail on IFP Functions

IFP comes with two manual pages that describe the various IFP library functions:

ifppilib(3)
ifplib(3)

The first of these manual pages describes the functions supplied to an IFP plugin. In general, the features of this library will not be of much interest, since the bulk of the functions in it are present to support general internal IFP operations.

The second library manual page describes functions such as ifp_manager_locate_plugin(), and is probably the more useful of the two. The libifp library contains functions that, notionally, operate on different classes of IFP objects. The manager level functions are likely to be the most useful, but you can interact with the library at all levels.

Manager functions

The two most immediately useful IFP manager functions are

ifp_manager_locate_plugin (const char *filename)
ifp_manager_run_plugin (ifp_pluginref_t plugin)

Together, they offer enough functionality to build a straightforward IF interpreter using plugins. Ifp_manager_locate_plugin() is responsible for finding, loading, and returning a plugin capable of handling a particular input file. Ifp_manager_run_plugin() then runs this plugin. There is also a form of ifp_manager_locate_plugin() that takes a URL for an IF game as its argument.

For example, here is a short code fragment that takes a URL or local file reference for a game, downloads the data into a temporary file if necessary, and then finds and runs the right interpreter engine for the game:

ifp_urlref_t       url;
ifp_pluginref_t    plugin;
url = ifp_url_new_resolve (url_path);
if (url != NULL)
    {
        plugin = ifp_manager_locate_plugin_url (url);
        if (plugin != NULL)
            {
                ifp_manager_run_plugin (plugin);
                ifp_loader_forget_plugin (plugin);
            }
        else
            {
                printf ("No plugin engine found for the data file\n");
            }
    }
else
    {
        printf ("Unable to find or resolve URL\n");
    }

There are two functions to manipulate the path that the manager searches for plugins:

ifp_manager_set_plugin_path (const char *new_path)
ifp_manager_get_plugin_path (void)

If no value is explicitly set. the manager uses, and returns, the path set in the environment variable IF_PLUGIN_PATH.

Loader functions

The IFP loader manipulates a list of loaded plugins. The functions

ifp_loader_search_plugins_dir (const char *dir_path)
ifp_loader_search_plugins_path (const char *load_path)

will search a directory and a path respectively, and load each IFP plugin they can find. If a particular plugin is already loaded, the loader will skip it on a search, ensuring that only one copy of each available plugin is loaded. The IFP manager uses these functions to refresh the list of loaded plugins each time its ifp_manager_locate_plugin() function is called.

The loader functions

ifp_loader_load_plugin (const char *filename)
ifp_loader_forget_plugin (ifp_pluginref_t plugin)

can be used to load an individual file as a plugin, and to remove a particular plugin from the loader, then delete it completely, freeing the memory it is using. To iterate round all the plugins in the loader, use

ifp_loader_iterate_plugins (ifp_pluginref_t current)

The following example code will find and load all plugins on the current manager path, and print out the name and version of the interpreter engine in them:

ifp_pluginref_t plugin;

ifp_loader_search_plugins_path (ifp_manager_get_plugin_path ());
if (ifp_loader_count_plugins () > 0)
    {
        for (plugin = ifp_loader_iterate_plugins (NULL);
                        plugin != NULL;
                        plugin = ifp_loader_iterate_plugins (plugin))
            {
                printf ("Found plugin %s-%s\n",
                                ifp_plugin_engine_name (plugin),
                                ifp_plugin_engine_version (plugin));
            }
    }
else
    {
        printf ("There are no plugins available\n");
    }

URL functions

A convenient way to resolve URLs is to use one of the following functions:

ifp_url_new_resolve (char *urlpath)
ifp_url_new_resolve_async (char *urlpath)

These calls create a new URL object, with the data downloaded into a temporary file if the URL is not a local file. "File:", "http:", and "ftp:" URLs are all acceptable, or you can also simply pass in a file path, for example /home/myfiles/somegame.z5.

If you use asynchronous URLs, the call will return as soon as the download of data (if remote) has started. In this case, the program can undertake other tasks, and you need to check periodically to see if the download has completed. To do this, use the functions

ifp_url_poll_status (ifp_urlref_t url, int *status)
ifp_url_poll_progress (ifp_urlref_t url)

If the URL has completed, ifp_url_poll_status() will return TRUE, with the completion status (a value of errno) in status. Otherwise, it will return FALSE. Ifp_url_poll_progress() returns the number of bytes downloaded so far for a remote URL; this is useful in printing download progress status messages. To pause while waiting for the next block of download data for a URL, call

ifp_url_pause (ifp_urlref_t url)

This returns on the next block of downloaded data, or after a short pause. You can set the pause length explicitly with the environment variable IFP_URLTIMEOUT_, giving the timeout in microseconds. Once the URL has finished downloading, it can be used in calls to other functions. The functions

ifp_url_get_url_path (ifp_urlref_t url)
ifp_url_get_data_file (ifp_urlref_t url)

return the path used to resolve a URL, and the file containing URL data, respectively.

The following code example shows how to resolve a URL asynchronously, displaying a status update as the data is downloaded:

ifp_urlref_t    url;
int             status;
url = ifp_url_new_resolve_async (urlpath);
if (url != NULL)
    {
        while (!ifp_url_poll_status (url, &status))
            {
                printf ("Busy downloading, %d bytes so far...\n",
                                ifp_url_poll_progress (url));
                ifp_url_pause (url);
            }
        if (status == 0)
            {
                printf ("URL is available for use.\n");
            }
        else
            {
                printf ("URL failed: %s\n", strerror (status));
                ifp_url_forget (url);
            }
    }
else
    {
        print ("Unable to find or resolve URL.\n");
    }

To free memory in use by a URL, call

ifp_url_forget (ifp_urlref_t url)

This will not, however, delete any temporary file associated with a remote URL. These files are kept in a cache in case they are needed again.

URL cache

Whenever IFP needs to download URL data into a temporary file, it keeps a record in an internal cache of the URL, and the temporary file containing the data. If another request is made for the same data, the same temporary file path is used, to save downloading the URL data multiple times. This makes it convenient to play a game more than once using a remote URL.

At present, this is a temporary cache; it exists only while the main IFP application is running. On program exit, all cached URL temporary files are deleted. The current default cache size is 10Mb; you can use the environment variable IFP_CACHELIMIT_ to set a cache size limit in bytes.

This feature is used by xlegion, but ifpe and xifpe will not get to take advantage of it since they exit after each game played.

Preferences functions

When the IFP manager begins to use an IF plugin, it must construct a set of startup options to pass to the interpreter's glkunix_startup_code() function. It does this by consulting a list that IFP keeps of the preferences to be used for particular plugins.

The two main functions to add and delete preferences for interpreter plugins are

ifp_pref_register (const char *engine_name, const char *engine_version, const char *preference)
ifp_pref_unregister (const char *engine_name, const char *engine_version, const char *preference)

These functions tell the IFP library how to construct a set of startup arguments when running a plugin. Preferences are added by ifp_pref_register(). Each call to this function needs to supply the name and version of the engine; IFP will match these to a plugin before starting to run it. The preference is simply a text string to be added to the glkunix_startup_code arguments. Preferences appear in the arguments in the same order as they are entered into the IFP library list. To register preferences for all versions of a plugin, the engine_version may be NULL. To enter global preferences, both engine_name and engine_version may be NULL.

Ifp_pref_unregister() removes a preference from the list. Engine_name and/or engine_version may be NULL, indicating wildcard values. Preference may also be NULL; a call with three NULL arguments will clear the entire list.

Initially, the preferences list is empty. Here is how IFP knows to add the "-ignore" argument each time it needs to start an instance of the Nitfol interpreter plugin:

ifp_pref_register ("nitfol", NULL, "-ignore");

IFP automatically passes preferences lists through chaining plugins, so a set of preferences written to the main program are used globally.

To find out which preferences a plugin understands, call the function

ifp_pref_list_arguments (const char *engine_name, const char *engine_version)

The function returns a glkunix_argumentlist_t result. To accomplish this, the function will load all available plugins into the loader, so if you're not expecting this, you might want to follow up with a call to ifp_loader_forget_all_plugins() when you have finished using the result of the ifp_pref_list_arguments() call.

Plugin functions

There are lots of functions for returning information about a particular plugin, such as its name, version, author, home page URL, and so on. Some of these, such as ifp_plugin_engine_name() and ifp_plugin_engine_version() used in the loader example above, may be of interest; others may be useful only in the implementation of IFP itself. See the libifp manual page for more details.

Notes on Building IFP Source

The main source for IFP lives in the /ifp subdirectory of the main IFP tree. This subdirectory contains its own Makefile, and leads a sort of independent existence within the IFP distribution. The source in this directory depends on Xglk having already been built, or at least unpacked (it needs the Glk header files). Once Glk (see below) is built, IFP will build, and run, given the limitation that until the plugins that do real work are built, there's little you can do with xifpe, ifpe, or xlegion that is useful.

To build the source, it's really necessary to build Glk before anything else. IFP needs to be built next, and the plugins can then be built in more or less any order. The top-level IFP Makefile does this; to build absolutely everything, it should be necessary to only type

make all

at the top of the unpacked IFP distribution.

The *.hdr files contain data definitions for the plugins. The files are filtered with ifphdr to produce 'C' source modules, which are then compiled and linked into the plugin's shared object.

The default plugin search path, if no value is set in the environment variable IF_PLUGIN_PATH, is "/usr/local/lib/ifp:/usr/lib/ifp". The IFP libraries complain if no value is set for IF_PLUGIN_PATH.

Tracing IFP actions

If you want to see what the IFP code is doing, you can turn on various levels of tracing output. Tracing is controlled by setting values for the environment variable IFP_TRACE_. Set this to contain the name of the IFP module or modules that you want to trace. For example, to trace IFP plugin loader functions, the value is "ifp_loader", and for both the loader and the plugin manager, "ifp_loader ifp_manager".

To quickly become neck-deep in trace and debugging information, set IFP_TRACE_ to "all".

Distributions known to work

This release of IFP has been verified to build on Caldera OpenLinux 3.1 and Red Hat Linux 7.1. There's nothing in it as far as I know that is distribution-specific, so it ought to be fine on other Linux's too. If you find that your's doesn't build IFP for some reason, please let me know.

Stripping the libraries and executables

Since it is a work-in-progress, IFP currently builds with full debugging on. This makes the plugins, libraries, and executable binaries much larger than they'd really need to be. Install IFP with the "make install-strip" option to get rid of the excess debugging data (and there is quite a lot of it).

Included subcomponents

This release of IFP includes a number of software components from outside of IFP in order to make it useful. These components live in the /interpreters subdirectory. They are direct, unmodified copies of the files held on IF Archive, the definitive source for such things. Any changes needed by IFP are shown (if this word can be applied to something that resembles modem line noise) in the *.patch diff-output files.

Likewise, the example games included in /games, and compressed into various formats in /derivatives, are also direct, unchanged copies of the game files from the IF Archive.

Portability

At the start of the project, I had the usual good intentions to make IFP portable. The shared object loading routines, and malloc and free stuff, are all wrappered, which may help to some extent. However, I pretty soon realized that I have no real idea of how shared object loading happens on platforms other than Linux, and that may of itself make the wrappers less than useful. When the time came to start adding sockets code to IFP, good intentions towards portability were quickly sacrificed on the altar of expediency (getting sockets code to work at all on Linux is always a chore, without adding the extra requirement of portability), and the use of SIGIO by IFP is almost certainly not immediately portable. The IFP 'porting layer' has really become just a convenient place to set debugger breakpoints.

If you know more about these areas on other architectures than I do (and this is almost certainly true), I'd be happy to hear suggestions about how to move IFP to other operating systems.

Building RPM packages

The source directory contains a spec file, for packaging IFP in RPM format. RPM package builds do work with IFP, though the RPM build makes only minimal effort to cut down on the debugging information that goes into a normal IFP build.

Notes on Running Games

The following notes might be useful when running games through the IFP binaries ifpe, xifpe, and xlegion.

Z-machine games

Z-machine game file names generally have either the extension .dat, typically used by an Infocom game, or .z1 to .z8, typically used by an Inform game. A Z-machine game is normally held in a single system file.

A compiled game encodes, in its first file byte, the version of the Z-machine specification for which it was built. In general, the number in an Inform extension reflects version number, but this is only a convention, and so the file extension might not always be accurate. Inform .dat extensions offer no clues about the Z-machine version that the game was compiled for.

The main Nitfol interpreter claims the ability to run all game versions 1 to 8, although the graphical features of version 6 may not be completely implemented. That said, it seems unwilling to run the Zork I and Zork II games, which are the only known version 2 games available. And there are, apparently, no version 1 games in existence. For this reason, the Nitfol interpreter plugin built for IFP restricts itself to game versions 3 to 8.

The Nitfol plugin accepts all valid Z-machine files, and can also handle Blorb encoded Z-machine games. Blorb Z-machine games have the executable resource type "ZCOD".

The IFP binaries always give Nitfol the "-ignore" option, to suppress any warnings about what it considers to be invalid game code.

TADS games

By convention, TADS game file names have the extension .gam, and a TADS game is a single system file.

The first few bytes of a TADS game file are "TADS2 bin", which make them very easy to identify, and therefore easy for IFP to route to the TADS interpreter plugin.

HUGO games

Normally, HUGO game file names have the extension .hex, and as with Z-machine and TADS games, the complete game is usually held in a single system file (a few HUGO games are built to include extra resources like images and sound, but this is somewhat rare at present, and IFP cannot currently handle it).

The IFP plugin for HUGO accepts games by looking for a date string, having the format mm-dd-yy, at byte offset three in a file.

Alan games

Alan games are normally distributed as a pair of files, having the same base file name, and the extensions .acd and .dat.

Alan .dat files have no discernable magic number or file identifier in them. Alan .acd files have the Alan version number in the first two bytes of the file. The Alan interpreter built for IFP can run Alan games at versions 2.7 and 2.8, so the plugin for Alan games looks for these two values in the version number bytes of a file.

When running Alan games with IFP, you should give the .acd file's name to the IFP binary, since this is the Alan game data file that has a recognizable signature in it.

Alan games are often packed into .zip archives, so a convenient way to run an Alan game is to simply point IFP at the .zip archive; IFP's Pkunzip plugin will automatically unzip the archive, then find the .acd file in it, and hand off control to the Alan plugin.

Glulx games

Glulx games are typically all in a single system file, and the file name, by convention, has the extension .ulx.

The first few bytes of a Glulx game file are always "Glul", and the Glulx plugin for IFP uses this to recognize games that it can interpret.

Glulx games can also be held in a Blorb file. In this case, the Glulx plugin will recognize a Blorb file having an executable resource type "GLUL".

AGT games

AGT games are normally distributed as a collection of eight or more related files. All files have the same base name, but there are multiple differing extensions: .da1, .da2, .da3, .ins, .d$$, and so on. There is also an alternative format that is special to AGiliTy, in which the complete game is held together in a single file with the extension .agx.

Single file .agx games are easily recognized by the AGiliTy interpreter plugin built for IFP, since they begin with the four bytes 0x58, 0xC7, 0xC1, and 0x51.

For multiple file games, the only file which has a recognizable signature inside it is the .d$$ file, so when running an AGiliTy game like this through IFP, this is the file name that you should give to the IFP binary.

As with Alan games, AGT games are often packed into .zip archives, so a convenient way to run an AGT game is to point IFP at the .zip archive, and have IFP's Pkunzip plugin automatically unzip the archive and find, and then run, the AGT game in it.

When running AGT games directly from .zip files, you may occasionally see a warning from IFP about a problem uncompressing a part of the archive. Typically, the error code will be 81. This is an indication that one or more of the files in the .zip archive could not be uncompressed -- normally this will be a "Shrunk" file. Linux's unzip command refuses to handle this compression type for legal reasons.

On seeing error 81 unzipping an archive, IFP's Pkunzip plugin will issue a warning, and continue. This normally allows the game to play. The files missing from the uncompression are usually either a "readme" text file, not used by the interpreter, or a .ttl game title file, and AGiliTy is usually able to run a game that is missing its title file. Although it's not helpful, AGiliTy may print the path to the game file it is running in place of the game title. To solve this annoyance you need to place an unzip program on your path that can handle "Shrunk" compression.

AGT games often assume that they are being displayed on 80x25, fixed font screens, so on occasions you may need to alter the display window dimensions when running such games to get them to display properly. Please see the AGiliTy Glk port readme notes for more information.

AdvSys games

AdvSys games are contained totally within a single system file. The few example games I've been able to locate use the extension .dat, or are distributed as a .dat file together with assorted supporting material in a .zip archive.

There is an element of filename confusion here, since Infocom games may have .dat file names, and to add to the confusion, Alan also has .dat files. IFP is unaffected by this since it ignores a game's file name. AdvSys files contain a well-defined and easily found signature.

Level 9 games

Level 9 games can come in a variety of formats, but the only one with anything close to a recognizable signature in it that the interpreter can understand is the Spectrum "snapshot" format, usually contained in a file with the extension .sna. Interpreters that understand this format generally implement a sequential scan through the memory snapshot data until they find the runnable part of the game (the Level 9 "A-code").

Though there is no defined .sna signature, observation shows that bytes at offsets one to four in the file are generally 0x58, 0x27, 0x9B, 0x36, so this is what the interpreter plugin will search for. If you have a .sna file containing a Level 9 game, and it matches this format, IFP binaries will be able to recognize and run it directly.

If however the data is in "raw" format, or if you have a .sna file that happens not to contain the signature that IFP is looking for, you can add an artificial signature - "LVL9" in the first four bytes - and the Level 9 plugin will find this signature and respond to it. To add this signature under Linux, you can use a command like

( echo "LVL9"; cat somefile.dat ) >somefile.lev

to do this. IFP binaries will now recognize the file "somefile.lev" as a Level 9 game file.

The IF archive contains a number of Level 9 games in snapshot format in the file level9.zip inside the Spectrum games subdirectory, and the Level 9 interpreter can play all games from version 2 to version 4. All of these game files have the .sna signature for which the IFP Level 9 plugin searches.

To manipulate other Level 9 game file formats, in particular .Z80, you might want to investigate the spconv tool, also available from the IF Archive.

Magnetic Scrolls games

By convention, Magnetic Scrolls games are held as a game text data file, usually having the extension .MAG, and an optional game pictures (graphics) file, with the extension .GFX. If both files exist, they should have the same base file name, and be held side by side in the same directory, so that Glk Magnetic can find them both correctly.

A game may offer an additional .HNT file, containing game hints. If available, this file should also have the same base name as the main game, and live in the same directory, so that Glk Magnetic can find it automatically. This may mean copying or renaming files on occasion. For example, the Magnetic Windows "Collection" games come with one hints file, COLL.HNT, so to make this work with Corruption, for example, you would need to copy COLL.HNT to CCORRUPT.HNT, and make sure that the files CCORRUPT.MAG, CCORRUPT.GFX, and CCORRUPT.HNT are all in the same directory.

The .MAG file has the signature that IFP looks for ("MaSc" in the first four bytes). If there is an associated graphics file, the Glk port will display pictures appropriate to the game. Otherwise, it will run in text mode only. If hints are available, then Glk Magnetic will use them, otherwise, it will run the game without hints.

Bugs and Omissions

Garbage-collection after running a game interpreter may not be complete, or sufficient. Especially when used in concert with chaining plugins, for some reason.

I'm suspicious of a memory leak in one or more of the xlegion components. In particular, playing a Z-machine multiple times seems to leak away about 100Kb on each iteration. At the moment, I don't know where this leak resides. Since plugins are garbage-collected, it's hard to see how the leak is an artifact of simply re-running the Nitfol plugin. But the effect seems more evident with this than any other plugins, which seems to suggest a connection to the specific plugin used, somehow. I'm currently considering a leak caused by the fact that Nitfol sets up a lot of Glk styles and hints, and these may not be getting cleared away correctly, either by Glk or by the IFP code.

Playing several compressed games in a row, or general extensive use of chaining plugins through xlegion caused early versions of it to become unglued, but changes since the first releases seem to have largely eliminated these problems.

I may have misunderstood several attributes of the Glk API. The global 'resetting' that IFP applies to Glk might be over-zealous. On the other hand, it might be insufficient. In particular, Glk seems to lack the capability to reset any set Blorb resource map to NULL -- once a program has set a resource map, it seems that the only way to get Glk to stop using that map is to hand it another Blorb resource map.

IFP is extremely sensitive to revisions or extensions to Glk. Since libifppi replicates every Glk function, any changes or additions to Glk will require the same changes to IFP.

URLs aren't really well integrated into the rest of IFP; they're a bit of a bolt-on. This is at least partly intentional. This way, there's no compulsion to wrap everything in the expense of a URL, and it is likely that better URL-handling library functions (for example, the QFtp class in libqt) exist elsewhere.

Keep your eyes open for temporary files left hanging about in /tmp/ifp_*. It's normal for such files to be present while IFP is running, but they should disappear when the program exits.

IFP needs a lot more testing. I've played about six moves or less into all the example games, and that's about it.

Goodness only knows what else.

Unfinished Business

This is just a partial list of the unfinished parts of IFP. I hope to be able to address them as time permits:

Where multiple plugins appear able to run a file, IFP selects only the first found. This prevents a program from being able to easily offer the ability to choose between two or more interpreter engines to handle the same game format (for example, Nitfol, Zip, Frotz...).
The only HTTP request status codes handled are 200 (success) and 404 (document not found). In particular, HTTP redirections should be possible.
FTP server status codes are not fully dissected or differentiated. In particular, the FTP response for where a file is absent is the same one as for where the file exists but there is no permission for the server to read it.
The URL cache persists only as long as the main program is running. It might be useful to make the cache persistent across program runs. Maybe even shared among all programs built with libifp. This raises the ugly spectre of checking cached data with the server, to detect updates since the last download. Not to mention the whole disk cache maintenance thing.
The internal workings of dup2 are unclear.
Have I got the acceptor patterns right for all of the game plugins? I somehow doubt it. In particular, I'm concerned about Inform and Hugo. It also seems that with Alan version 3 there's the potential for a clash with the current Inform pattern.
Does the whole game acceptor pattern matching thing need to be made much smarter and more general purpose? Multiple patterns, with AND and OR operations?
Chaining plugins are a real nuisance with the Linux dynamic loader. If it spots that a process has already loaded a particular file, checked by inode number, it will return the same handle to the shared object, and increment the object's reference count. This is probably all fine and good for real shared library work, where the aim is to save system memory, but for IFP purposes when we ask for a new instance of a plugin, we truly want a new instance, otherwise IFP cannot nest file uncompression (I'd live with shared text space and separated data -- believe it or not, UNIX offers this). The current solution is to copy (yes, really copy) all but the first instance of a chaining plugin, and the chosen plugin to chain to inside a chaining plugin, to a temporary file, and load that file. This is not as bad as it sounds, since it doesn't need to happen often. But it's untidy, and inefficient, when it does happen. It's also very hard to spot when it has gone wrong.
The pkunzip and tar plugins search sequentially through the top level of the archive they just unpacked, and play only the first playable game file that they find. This may be good enough for most practical purposes. On the other hand, it may not be.
Some of IFP's internal data structures are singly linked lists, and would be better as doubly linked lists. Likewise, there may be scope for improvements in some places with hash tables (although, apart from malloc addresses for garbage collection, which are already in hash tables, in practice IFP doesn't tend to keep many data structures large enough to benefit much from this, I don't think).
IFP resets various things inside of Glk without really knowing what it is doing. Glk seems to be able to withstand this, which is great. However, I may have abused, or misused, this feature.
URL parsing may be wrong, either slightly or completely.
I don't fully understand the nuances of Glk's restriction on only being able to call the function glkunix_stream_open_pathname() from an interpreter's glkunix_startup_code(). The fact that a plugin is garbage-collected might really interfere with something about this function in Glk.
IFP should probably be made to work with the "Glk loader" (the thing that gives you runtime control over whether you get the curses Glk, the X Glk, or whatever). This will truly reduce the actual game-playing program to a totally invisible abstraction.
I can't help but think that there might be a better way to intercept malloc(), free(), and its cousins in a plugin other than the way IFP currently does things.
IFP sets up some signal handlers, mostly so it can delete temporary files when the application exits through something like SIGTERM or SIGINT. However, by the time it gets the chance to set the handlers, Glk has already called dibs on them. To try to cope, IFP saves the addresses that Glk put in the handlers, then installs its own. On receiving one of these signals, IFP will call the original handler before performing its own signal-handling dance. Is there a better way to handle this?
The way in which the registered options data is passed forwards through chaining plugins might benefit from another look. Perhaps other parts of IFP could use this same idea. Or perhaps the idea is not a good one, and options should work in some other fashion.
The whole interpreter options area is incomplete, anyway.
It wasn't clear to me whether the Glk dispatch interface would help with IFP. It seemed that no matter what, IFP would have to provide the bulk of Glk symbols for itself, so that interpreters can link to libifppi.a. It would be possible inside libifppi.a to call Glk dispatch rather than make the function call through the (rather large) Glk interface structure. However, there didn't seem to be much to be gained from this except for shrinking the size of the Glk interface structure, and since this is hidden from an IFP client program and totally internal to libifppi and libifp, I wasn't sure that there was too much point in doing this. The Glk interface structure would still have to contain the dispatch and Blorb functions, even with this change.
Is there any sense in a plugin that can dig games out of Apple II disk images? Or one that can find and run the game text part of a DOS exe IF game?
I'd like IFP to handle the thorny issue of how to offer hooks for games programs to control things like saving and restoring games, restarting or quitting the current game, and so on. For example, a floppy disk icon button could save the current game. Though it sounds simple, it seems to be more complex than I'd imagined. This is because saving the game is implemented differently, as far as I can tell, in each interpreter. And worse, sometimes it seems to reside in the game itself. There is no standard method, as far as I can tell. In then end, if doable at all, all I can think of for this feature is some form of glorified "macro" facility, where pushing the "save game" button injects the string "save" into the game's input. Not all that great.
IFP does nothing to support the possibility of using threads to run multiple games simultaneously. Not as odd as it sounds -- I use a bunch of game hints that all come in Z-machine format, and it's nice to have both hints and the game they hint at running at the same time.
Even in the presence of threads, IFP needs a way to be able to run multiple plugins at the same time. This would probably take the form of augmenting Glk calls that a plugin makes through the Glk interface with some form of "session ID" -- this might be the plugin's own identifier, or a Glk identifier for a multi-session Glk library, and would allow the IFP handlers to de-multiplex incoming Glk calls, and route each call to the relevant Glk session. At present, however, no multi-session Glk library implementations exist, so for now this remains a vague idea.
The Xglk library should be built with sound support. I did experiment with the sound patches, but couldn't get any of them to work with the (admittedly very, very few) IF games I have that use sound. I stubbornly got either complaints from sox about "Bogus AIFF file: MARKers but no INSTrument.", or silence. For now, then, there is no sound support compiled into Xglk in IFP. I did however apply the 24-bit display patch.
IFP needs a really cute Icon.

Acknowledgments

The current distribution of IFP contains the following code elements from other sources:

Xglk 0.4.11, Glkterm 0.7.8, and Cheapglk 0.8.7. These are X, curses, and standard I/O implementations of the excellent Glk library from Andrew Plotkin. Without this, IFP would be impossible. IFP doesn't make any changes to Glk beyond standard patches for controlling the build and correcting colour on 24-bit X servers and a small patch to fix eight bit character input in Xglk -- it works totally within the framework of the Glk API. Versions of Xglk, Glkterm, and Cheapglk are included for convenience only.
Nitfol 0.5. A plugin scheme is no use without plugins to interpret games. IFP includes a copy of Evin Robertson's Nitfol interpreter for Z-machine games. The files nitfol_es.patch, nitfol_debug.patch and nitfol.patch show the extent of the changes that I made to the Nitfol source. The first patch fixes a bug in Nitfol that prevents eight bit characters from displaying correctly in Spanish games. The second fixes a handful of null pointer dereferences in Nitfol's game debugging code. The third contains the changes that make Nitfol build as a plugin; this one is all Makefile changes.
Alan 2.8.6, from Thomas Ljungsbro. The only changes needed to turn this into an IFP plugin are in the Makefile.
Glulxe 0.3.5. This is Andrew Plotkin's own interpreter for an alternative format for Inform compiled games. The only changes are in the Makefile.
He 3.0. This is Kent Tessman's Hugo Engine. Again, the only changes are in the Makefile.
Tadsr 2.5.7. Michael J. Roberts' TADS interpreter. As with Alan, Glulxe, and Hugo above, the interpreter only requires Makefile changed to build as an IFP plugin.
AGiliTy 1.1.1. This is Robert Masenten's AGT interpreter. The Glk porting layer of this interpreter is mine, and was built after IFP was conceived, so there are barely any changes to make in order to integrate it fully into IFP.
AdvSys 1.2. This is David Betz's Adventure Writing System interpreter. Michael Chen wrote the Glk porting code for the original AdvSys 1.2. I added the capability to use a command line argument as a game file, in the file advsys_argv.patch, then made the usual Makefile changes to build as a plugin, in the file advsys.patch.
Level9 3.0. This is Glen Summers' interpreter for Level 9 games in any format, including Spectrum snapshots. David Kinder has also done considerable work on this interpreter. The Glk porting layer of this interpreter is mine, so it requires virtually no change to integrate it into IFP.
Magnetic 2.1. Niclas Karlsson's interpreter for Magnetic Scrolls and Magnetic Windows games. As with Level9 above, the Glk porting layer for Magnetic Scrolls is mine. It needs only the most minimal of changes to integrate it into IFP.

As a general rule, any changes I made to any of the above will be found in a .patch file in the IFP source distribution. The plugin header definitions for pluginifying the various interpreters is in the *.hdr files. I hope I got the details right in each -- if I made any errors, please drop me a piece of email and I'll fix it right away.

The distribution also includes the following example games:

Bugged, Copyright by Anssi Raisanen.
Adventure (The Interactive Original), by Will Crowther and Don Woods, reconstructed by Donald Ekman, David M. Baggett, and Graham Nelson.
Lords of Time, Copyright Level 9 Computing.

I'm extremely grateful to the authors of all of the interpreters and games included in IFP. My only reason for including them directly in the current IFP distribution is convenience. With them all bundled up in one archive with IFP, it is easy to build and test a complete multi-game interpreter quickly, without needing to hunt around and find other required components, and the consequent risk of finding a mismatched version of one or the other. All of these included items packaged in IFP are unmodified from their original format, and all are available in one form or another from the IF Archive.

I've checked the copyrights and blurb on each of the items carefully, and I'm fairly convinced that it's perfectly okay to include them in IFP in this way. If you are an author of one of the above items, and you don't want your work included in IFP for some reason, please let me know, and I will remove it immediately.

Because Magnetic Scrolls games are still copyrighted, the IFP distribution does not include an example Magnetic Scrolls or Magnetic Windows game.

Thanks to David Griffith, Matthew Krauss, Bob Newell, and Curt Siffert for their feedback, helpful comments, and ideas, following the first Alpha release of IFP, and to Gregory W. Kulczycki for regular builds of IFP for Mandrake Linux.

Appendix A - Strdup, Optimization, and IFP

The GNU gcc strdup() function, when compiled with optimizations on, contains an unfortunate construct that causes problems with IFP. The problem happens because IFP, in trying to garbage collect interpreters, needs to intercept and register each and every call that the interpreter makes to allocate heap memory, and also all the calls to free that same memory.

With normal strdup() calls, there is no problem. However, the GNU header file /usr/include/bits/string2.h, included from the main string.h, contains optimizations for string functions where the -O compiler option is given. The optimization for strdup() transforms a simple 'strdup("hello");' statement into:

(__extension__ (__builtin_constant_p ("hello") && ((size_t)(const void *)(("hello") + 1) - (size_t)(const void *)("hello") == 1) ? (((__const char *) ("hello"))[0] == '\0' ? (char *) calloc (1, 1) : ({ size_t __len = strlen ("hello") + 1; char *__retval = (char *) malloc (__len); if (__retval != ((void *)0)) __retval = (char *) memcpy (__retval, "hello", __len); __retval; })) : __strdup ("hello")));

In and of itself, this is scary enough. However, the problem part for IFP is that the macro definition of strdup() in string2.h has converted the statement from a call to a strdup() function into a call to __strdup(). IFP can only intercept calls to strdup(), since these are defined as weak symbols in glibc. It can't catch or redefine __strdup(), since __strdup() is not a weak glibc symbol.

The result of this is that if an interpreter uses strdup() and is compiled with optimizations, IFP doesn't get to see the __strdup() call. However, the subsequent free() of the address returned by string duplication is caught by IFP. As far as IFP is concerned, this looks like an attempt to free memory that hasn't been allocated from the heap, and the result is a fatal IFP error.

To avoid this, if an interpreter uses strdup(), either compile it without optimizations, or disable the optimizations in string2.h by passing the option -D__NO_STRING_INLINES to the gcc compiler.

It is debatable whether the GNU optimization of strdup(), bypassing as it does the weak symbol definition in glibc, is a bug, or just a highly aggressive and unhelpfully implemented feature.

Appendix B - X Resources for a More Attractive Xglk

After a bit of experimentation, I came up with a set of X resources that make Xglk look a little more attractive on my system. It's nothing revolutionary, but I prefer the look of Xglk with these resources in place to its rather more drab default appearance, and you may too.

For what it's worth, then, here are the X resources I generally use for Xglk. These are held in my ~/.Xdefaults file, and for convenience, a copy of this file is shipped as documentation in the IFP binary distributions.

Glk.geometry:           600x700
Glk.foreground:         black
Glk.background:         WhiteSmoke
Glk.linkColor:          blue
Glk.selectColor:        SteelBlue
Glk.frameColor:         grey50
Glk.frameUpColor:       grey75
Glk.frameDownColor:     grey25
Glk.ditherImages:       true
Glk.colorLinks:         true
Glk.underlineLinks:     true
Glk.promptDefaults:     true
Glk.historyLength:      20
Glk.textBuffer.saveLength:      16000
Glk.textBuffer.saveSlack:       1000
Glk.textGrid.foreground:        black
Glk.textBuffer.foreground:      black
Glk.textGrid.background:        WhiteSmoke
Glk.textBuffer.background:      WhiteSmoke
Glk.Window.linkcolor:           blue
Glk.Window.margin.x:            4
Glk.Window.margin.y:            2
Glk.Window.Style.size:          0
Glk.Window.Style.weight:        0
Glk.Window.Style.oblique:       0
Glk.Window.Style.proportional: 1
Glk.Window.Style.indent:        0
Glk.Window.Style.parIndent:     0
Glk.Window.Style.justify:       1
Glk.Window.Style.foreground:    black
Glk.Window.Style.background:    WhiteSmoke
Glk.Window.Style.linkColor:     blue
Glk.Window.alert.foreground:            red1
Glk.Window.input.foreground:            DarkRed
Glk.Window.input.weight:                1
Glk.Window.header.foreground:           DarkGreen
Glk.Window.header.weight:               1
Glk.Window.header.size:                 +1
Glk.Window.subheader.foreground:        DarkGreen
Glk.Window.subheader.weight:            1
Glk.Window.emphasized.foreground:       DarkGreen
Glk.Window.emphasized.oblique:          1
Glk.Window.preformatted.proportional:   0
Glk.textGrid.Style.proportional:        0
Glk.textGrid.Style.foreground:          NavyBlue
Glk.textGrid.Style.Spec:        -b&h-lucidatypewriter-\
%w{medium,bold}-%o{r,r}-normal-*-%s{8,10,12,14,18,24}-*-*-*-*-*-iso8859-1
Glk.textBuffer.Style.Spec:      %p\
{-b&h-lucidatypewriter-\
%w{medium,bold}-%o{r,r}-normal-*-%s{8,10,12,14,18,24}-*-*-*-*-*-iso8859-1,\
-adobe-times-\
%w{medium,bold}-%o{r,i}-normal--%s{8,10,12,14,18,24,34}-*-*-*-*-*-iso8859-1}

IFP Copyright

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

This distribution of IFP also contains code from other source. For copyright information on the included code, please see the relevant copyright or similar files in their archive or distribution files.