NATO.0+55+3d modular, page 08/14
Image Processing
Image Processing Objects
So, this is the fun stuff. The image processing objects available in NATO are very likely the centerpiece of any live video programming you're going to be doing. As with everything in NATO, they work in real time, and process incoming image data (playing from disk, or digitized on the fly) fairly efficiently (depending on the object, of course).
Given the number of processing objects, it would take far too long to go through each of them (more than half of NATO's objects are for processing images), so I'm going to limit this section to a brief discussion.
The processing objects range from the practical to the whimsical. They encompass everything from band-pass and -reject filters, to gain controls, to blur and sharpen objects, to image swirlers and noise synthesizers. Many of the effects bundled with NATO resemble basic Photoshop filters, and are only useful to a point, unless you really like that sort of thing. I've ended up writing my own effects (in C) and using them in preference to the built-in processing, for the most part.
As far as speed goes, NATO's image processors range from very efficient to very slow. Basic point/color processors are fairly quick (bandpass, bandreject, invert, etc.). More complex processors (blur, edge detect, displace) are quite a bit slower, and some processors are really unusable in real-time situations, at least on current hardware.
Each object takes multiple commands, only one or two of which are shown in the example above. Using many processing objects will eat up processor cycles and can drop your framerate down to a crawl. When building effect chains, you'll want to arrange it so that you can easily bypass objects you're not using. Nearly every NATO object has a `bypass` command which causes signals to pass through without change, so that's one method. Certain objects, though, such as the compositing objects (the `242.collage` series of objects), appear to slow processing down, even when bypassed using the `bypass` command. After a lot of trial and error, I've ended up building a network of gates that completely avoids unused objects and routines. This seems to work pretty well. Using only one or two processing objects at a time is completely practical, even with a lot of other Max activity going on, such as digitizing and recording.
[Technical side note: When image data is received by NATO objects, an offscreen graphics world is created in RAM. The incoming frame is copied to the gworld, and then read, analyzed, operated upon and written to an output buffer, before it's sent out of the outlet. This happens for each frame. When an object is bypassed, the processing function generally returns without any action being taken. However, variables are still declared, and the incoming frame is fed into a temporary buffer. Although the overhead may be minimal, with several objects in series, it can add up, even with all objects bypassed.]
Another usage of the image processing objects is demonstrated below. This example uses a couple of the channel operations objects supplied with NATO. In this case, the film is split into its rgb color channels using the `242.channel` object bank on the top left. These channels are reintegrated on the right using the `242.channel3` object. The `inlet $1 $2` message is turning the channel inlets off and on in response to random numbers. When the inlet turns off, the channel image `freezes` , causing the video channels to `slide` against each other when they reintegrate, since the channel information is being updated at different times. You can see the output of this patch here.
all materials on this site (text, images, etc.) © 2000-2001 Jeremy Bernstein