Working with Images

This chapter shows you how to load and render images using the Neutrino image library libimg.

GF doesn't have any image-handling functionality, aside from blitting images once they've been opened, decoded, and rendered to a surface. Instead, GF applications should use a separate library to accomplish these tasks.

The Neutrino image library is a static library that provides a common interface for image codecs. This means that while the library is linked into your executable, you need to put any required image codecs, plus the image configuration file, onto targets running your application. You'll need to include (at least):

the configuration file at /etc/system/config/img.conf
the image codecs (img_codec_*.so) in processor/lib/dll

The codecs used by the Neutrino image library are:

img_codec_bmp.so

Windows Bitmap format codec. Provides full Microsoft BMP support for all known variants as well as the older OS/2 variant. Does not support v1.x DDB format.

img_codec_gif.so

Graphics Interchange Format codec. Supports GIF 87a and GIF 89a variants. This codec supports the graphics control extension which provides most of the significant features for this format (transparency, interlacing, multiframe etc). It ignores other extensions which allow embedded text, comments, application data etc.

The img_codec_gif.so supports decoding, but not encoding due to patent restrictions.

img_codec_jpg.so

Joint Photographic Experts Group file format codec. Supports 24-bit RGB, YUV and grayscale.

img_codec_png.so

Portable Networks Graphics codec. Provides full PNG support with alpha, transparency, and interlacing support. This codec ignores the following chunks:

background color
chromaticity
gamma
histogram
physical pixel dimension
significant bits
text data
image last-modified time

img_codec_sgi

SGI format codec. It supports black-and-white, grayscale, and color images (*sgi, *.rgb, *.rgba, *.bw).

This codec only supports decode.

img_codec_tga.so

Truevision Graphics Adapter format codec. For decoding, this codec handles run length encoding (RLE) compression and supports these formats:

true color 16-, 24-, and 32-bit
cmap 15-, 16-, 24-, and 32-bit
black and white 8-bit

For encoding, this codec supports true color (8888) 32-bit RLE.

For a sample buildfile that includes a GF application and the required image codecs, see the Embedding GF chapter.

Code examples in this chapter are taken from the img_decode_simple example application shipped with QNX Advanced Graphics. This application loads an image into system RAM in a format that GF can display. See the QNX Advanced Graphics source package for full application source.

Before your GF application can render images, it must perform all the steps required to prepare for rendering, as outlined in the “Setting up GF” section of the Basic Drawing chapter.

To to display an image, your application also needs to:

attach to the image library
load the image file
attach to a GF surface for the image
blit the image onto the target surface
clean up allocated resources and detach from the GF surface and image library

Let's look at each of these steps in a little more detail.

Attaching to the image library

When you call img_lib_attach(), the image library initializes and loads the codecs it finds listed in the img.conf configuration file. You can customize this file to load just the codecs your application requires, and change the location where the image library looks for it by setting the LIBIMG_CFGFILE environment variable. By default, the library checks the default location /etc/system/config/img.conf. See The img.conf Configuration File for more information about the format of this file.

To use img_lib_attach():

img_lib_t ilib = NULL;
int rc;
...
if ((rc = img_lib_attach(&ilib)) != IMG_ERR_OK) {
                fprintf(stderr, "img_lib_attach() failed: %d\n", rc);
                return -1;
        }

Loading the image

Loading an image involves these steps:

Enumeration of codecs

First, you need a list of codecs that are installed, which you can retrieve by calling img_codec_list().

If you have additional information about the data (for example, a mime-type or extension), you could use a variant such as img_codec_list_byext() or img_codec_list_bymime(). This will give you a list of codecs including only those that match the specified criteria. Keep in mind though, that extensions or mime types do not necessarily guarantee the data is of a specific format (that is, they can lie). So it's always good to be prepared to try all the codecs if one that handles the format that data claims to be in fails.

Establish the underlying input source
The image data has to come from a source, such as a file, TCP/IP socket, or memory buffer. This step involves establishing the origin of the data. This step may involve no work at all (that is, if it's a file already stored in memory), or it may involve opening a file or performing some other task.
Associate the image library conventional IO interface with the input source
The image library decoders need a conventional way to access the data, which is where the IO streams come in. Use io_open() to associate an io_stream_t with the data source from the previous step.
Data recognition
This step involves allowing the codecs you've enumerated to peek at the data to determine which one is capable of handling the data. You can do this with img_decode_validate(), which runs through the list of codecs and indicates which (if any) approved of the data. You can then use that codec to decode the data.
Initialize the decoder
This step notifies the decoder of an imminent decode operation, and allows it to set up any resources it may require. Use img_decode_begin() to perform this step.
Decode frames
Decode frames using img_decode_frame() until you're finished or there are no more (when img_decode_frame() returns IMG_ERR_NODATA).
Finalize Decode
Call img_decode_finish() to allow the decoder to clean up after itself.

Although this process may seem complicated, there are two higher-level API calls that simplify the process:

img_load_file(): This function takes care of all of the steps outlined above. However, this function loads only the first frame, and works only with a file source.
img_load(): This function takes care of all the steps, except for establishing the input source and associating it with an io_stream_t. This provides the convenience of img_load_file() but lifts the file only restriction. Like img_load_file(), it is limited to loading only the first frame.

Here's an example of using img_load_file():

int rc;
img_t img;

...

/* initialize an img_t by setting its flags to 0 */
img.flags = 0;

/* if we want, we can preselect a format (ie force the image to be
   loaded in the format we specify) by enabling the following two
   lines */

img.format = IMG_FMT_PKLE_ARGB1555;
img.flags |= IMG_FORMAT;

/* likewise, we can 'clip' the loaded image by enabling the following */

img.w = 100;
img.flags |= IMG_W;

img.h = 100;
img.flags |= IMG_H;

if ((rc = img_load_file(ilib, argv[optind], NULL, &img)) != IMG_ERR_OK) {
        fprintf(stderr, "img_load_file(%s) failed: %d\n", argv[optind], rc);
        return -1;
}

fprintf(stdout, "img is %dx%dx%d\n", img.w, img.h, IMG_FMT_BPP(img.format));

/* for our purposes we''re done with the img lib */
img_lib_detach(ilib);

Attaching to a surface

This step is more involved. First, you need to determine whether the image is palette-based, and set the palette accordingly :

gf_palette_t palette;
...
if (img.format & IMG_FMT_PALETTE) {
        /* setup palette if necessary */
        palette.ncolors = img.npalette;
        palette.colors = img.palette;
} else if (img.format == IMG_FMT_G8) {
        /* we can render G8 images in GF by using a palette of grays */
        palette.ncolors = 256;
        palette.colors = (img_color_t*)g8pal;
}

Next, you can wrap the image in a GF surface, so that it can be used with the rest of the GF API. To do this, use gf_surface_attach(), passing the image's format, palette (if applicable), dimensions, stride, and image data:




/* attach a surface to the image data; this allows us to blit the image
   data to another surface (or the display) in GF */
if ((rc = gf_surface_attach(&img_surf, setup.gdev,
        img.w, img.h, img.access.direct.stride, img_fmt_to_gf(img.format),
        &palette, img.access.direct.data, 0)) != GF_ERR_OK) {
        fprintf(stderr, "gf_surface_attach() failed: %d\n", rc);

        /* might fail here if the decoder gave us a format that cannot
           map to GF; in this case we could have preselected a format
           that is supported by GF (code above shows how to do this) */

        return -1;
}

Note that a surface created with gf_surface_attach() isn't like other GF surfaces; there are some restrictions:

Any rendering or blitting to or from this surface is done in software, not hardware. This is because the memory isn't allocated by the video driver, so the driver doesn't know anything about it.
For that same reason, your application is responsible for freeing the image data, However, it should do so only after freeing the attached surface by calling gf_surface_free().
This surface can't be shared with other processes because it isn't managed by io-display.

Blitting the image

Before you blit the image onto a target surface, you should check to see if there's a transparency flag or alpha channel:

gf_setup_t setup;
...
if (img.flags & IMG_TRANSPARENCY) {
        /* we can handle transparency in GF using chroma */

        gf_chroma_t        chroma;
        memset(&chroma, 0, sizeof chroma);
        chroma.mode = GF_CHROMA_OP_SRC_MATCH | GF_CHROMA_OP_NO_DRAW;
        if (img.format & IMG_FMT_PALETTE) {
                chroma.color0 = img.palette[img.transparency.index];
        } else if (IMG_FMT_BPP(img.format) < 24) {
                chroma.color0 = img.transparency.rgb16;
        } else {
                chroma.color0 = img.transparency.rgb32;
        }

        gf_context_set_chroma(setup.context, &chroma);
}

if (img.format & IMG_FMT_ALPHA) {
        gf_alpha_t        alpha;
        memset(&alpha, 0, sizeof alpha);
        alpha.mode = GF_ALPHA_M1_SRC_PIXEL_ALPHA | GF_BLEND_SRC_M1 |
            GF_BLEND_DST_1mM1;
        gf_context_set_alpha(setup.context, &alpha);
}

You can blit the image onto a target surface by calling gf_draw_blit2():

 gf_draw_blit2(setup.context, img_surf, NULL,
                0, 0, img.w - 1, img.h - 1, setup.x1, setup.y1);

Cleaning up

Once you've finished using the image library, you should clean up by unlocking the hardware, disable the alpha and chroma, release the GF surface, and free the part of the image structure that you're responsible for:



/* it's a good idea to do this before we free the image to ensure the
   renderer is done with the data */
gf_draw_finish(setup.context);

/* unlock the h/w */
gf_draw_end(setup.context);

if (img.format & IMG_FMT_ALPHA) {
        gf_context_disable_alpha(setup.context);
}

if (img.flags & IMG_TRANSPARENCY) {
        gf_context_disable_chroma(setup.context);
}

/* release the attached surface; we're not going to blit from it any
   more (in real life this surface could be recycled if needed) */
gf_surface_free(img_surf);

/* above code only releases the surface; we still have the actual image
   data hanging around. We relied on the library to allocate this for
   us. The following is all that's needed when we rely on this default
   behaviour (note that this free() takes care of the palette also,
   if applicable, since the lib allocates it all in one chunk) */
free(img.access.direct.data);