Custom Image Processing in Java

Custom BufferedImageOp

Creating a new filter from scratch is not a very complicated task. To prove it, we show you how to implement a color tint filter. This kind of filter can be used to mimic the effect of the colored filters photographers screw in front of their lenses. For instance, an orange color tint filter gives a sunset mood to a scene, while a blue filter cools down the tones in the picture.

You first need to create a new class that implements the BufferedImageOp interface and its five methods. To make the creation of several filters easier, we first define a new abstract class entitled AbstractFilter. As you will soon discover, all filters based on this class are nonspatial, linear color filters. That means that they will not affect the geometry of the source image and that they assume the destination image has the same size as the source image.

The complete source code of our custom BufferedImage is available online.

Base Filter Class

AbstractFilter implements all the methods from BufferedImageOp except for filter(), which actually processes the source image into the destination and hence belongs in the subclasses:

public abstract class AbstractFilter
implements BufferedImageOp {
public abstract BufferedImage filter(
BufferedImage src, BufferedImage dest);
public Rectangle2D getBounds2D(BufferedImage src) {
return new Rectangle(0, 0, src.getWidth(),
src.getHeight());
}
public BufferedImage createCompatibleDestImage(
BufferedImage src, ColorModel destCM) {
if (destCM == null) {
destCM = src.getColorModel();
}
return new BufferedImage(destCM,
destCM.createCompatibleWritableRaster(
src.getWidth(), src.getHeight()),
destCM.isAlphaPremultiplied(), null);
}
public Point2D getPoint2D(Point2D srcPt,
Point2D dstPt) {
return (Point2D) srcPt.clone();
}
public RenderingHints getRenderingHints() {
return null;
}
}

The getRenderingHints() method must return a set of RenderingHints when the image filter relies on rendering hints. Since this will probably not be the case for our custom filters, the abstract class simply returns null.

The two methods getBounds2D() and getPoint2D() are very important for spatial filters, such as AffineTransformOp. The first method, getBounds2D(), returns the bounding box of the filtered image. If your custom filter modifies the dimension of the source image, you must implement this method accordingly.

The implementation proposed here makes the assumption that the filtered image will have the same size as the source image.

The other method, getPoint2D(), returns the corresponding destination point given a location in the source image. As for getBounds2D(), AbstractFilter makes the assumption that no geometry transformation will be applied to the image, and the returned location is therefore the source location.

AbstractFilter also assumes that the only data needed to compute the pixel for (x, y) in the destination is the pixel for (x, y) in the source.

The last implemented method is createCompatibleDestImage(). Its role is to produce an image with the correct size and number of color components to contain the filtered image. The implementation shown in the previous source code creates an empty clone of the source image; it has the same size and the same color model regardless of the source image type.

Color Tint Filter

The color tint filter, cleverly named ColorTintFilter, extends AbstractFilter and implements filter(), the only method left from the BufferedImageOp interface. Before we delve into the source code, we must first define the operation that the filter will perform on the source image. A color tint filter mixes every pixel from the source image with a given color. The strength of the mix is defined by a mix value. A mix value of 0 means that all of the pixels remain the same, whereas a mix value of 1 means that all of the source pixels are replaced by the tinting color. Given those two parameters, a color and a mix percentage, we can compute the color value of the destination pixels:

dstR = srcR * (1 – mixValue) + mixR * mixValue
dstG = srcG * (1 – mixValue) + mixG * mixValue
dstB = srcB * (1 – mixValue) + mixB * mixValue

If you tint a picture with 40 percent white, the filter will retain 60 percent (1 or 1 – mixValue) of the source pixel color values to preserve the overall luminosity of the picture.

The following source code shows the skeleton of ColorTintFilter, an immutable class.

public class ColorTintFilter extends AbstractFilter {
private final Color mixColor;
private final float mixValue;
public ColorTintFilter(Color mixColor, float mixValue) {
if (mixColor == null) {
throw new IllegalArgumentException(
"mixColor cannot be null");
}
this.mixColor = mixColor;
if (mixValue < 0.0f) {
mixValue = 0.0f;
} else if (mixValue > 1.0f) {
mixValue = 1.0f;
}
this.mixValue = mixValue;
}
public float getMixValue() {
return mixValue;
}
public Color getMixColor() {
return mixColor;
}
@Override
public BufferedImage filter(BufferedImage src,
BufferedImage dst) {
// filters src into dst
}
}

The most interesting part of this class is the implementation of the filter() method:

@Override
public BufferedImage filter(BufferedImage src,
BufferedImage dst) {
if (dst == null) {
dst = createCompatibleDestImage(src, null);
}
int width = src.getWidth();
int height = src.getHeight();
int[] pixels = new int[width * height];
GraphicsUtilities.getPixels(src, 0, 0, width,
height, pixels);
mixColor(pixels);
GraphicsUtilities.setPixels(dst, 0, 0, width,
height, pixels);
return dst;
}

The first few lines of this method create an acceptable destination image when the caller provides none. The javadoc of the BufferedImageOp interface dictates this behavior: “If the destination image is null, a BufferedImage with an appropriate ColorModel is created."

Instead of working directly on the source and destination images, the color tint filter reads all the pixels of the source image into an array of integers. The implications are threefold. First, all of the color values are stored on four ARGB 8-bit components packed as an integer. Then, the source and the destination can be the same, since all work will be performed on the array of integers. Finally, despite the increased memory usage, it is faster to perform one read and one write operation on the images rather than reading and writing pixel by pixel. Before we take a closer look at mixColor(), where the bulk of the work is done, here is the code used to read all the pixels at once into a single array of integers:

public static int[] getPixels(BufferedImage img,
int x, int y,
int w, int h,
int[] pixels) {
if (w == 0 || h == 0) {
return new int[0];
}
if (pixels == null) {
pixels = new int[w * h];
} else if (pixels.length < w * h) {
throw new IllegalArgumentException(
"pixels array must have a length >= w*h");
}
int imageType = img.getType();
if (imageType == BufferedImage.TYPE_INT_ARGB ||
imageType == BufferedImage.TYPE_INT_RGB) {
Raster raster = img.getRaster();
return (int[]) raster.getDataElements(x, y, w, h, pixels);
}
return img.getRGB(x, y, w, h, pixels, 0, w);
}

There are two different code paths, depending on the nature of the image from which the pixels are read. When the image is of type INT_ARGB or INT_RGB, we know for sure that the data elements composing the image are integers. We can therefore call Raster.getDataElements() and cast the result to an array of integers. This solution is not only fast but preserves all the optimizations of managed images performed by Java 2D.

When the image is of another type, for instance TYPE_3BYTE_BGR, as is often the case with JPEG pictures loaded from disk, the pixels are read by calling the BufferedImage.getRGB(int, int, int, int, int[], int, int) method. This invocation has two major problems. First, it needs to convert all the data elements into integers, which can take quite some time for large images. Second, it throws away all the optimizations made by Java 2D, resulting in slower painting operations, for instance. The picture is then said to be unmanaged. To learn more details about managed images, please refer to the Chapter 5, “Performance."

int[] getRGB(int startX, int startY, int w, int h,
int[] rgbArray, int offset, int scansize)

This method is used to retrieve an array of pixels at once, and invoking it will punt the optimizations made by Java 2D. Consider the second variant of getRGB():

int getRGB(int x, int y)

This method is used to retrieve a single pixel and does not throw away the optimizations made by Java 2D. Be very careful about which one of these methods you decide to use.

The setPixels() method is very similar to getPixels():

public static void setPixels(BufferedImage img,
int x, int y,
int w, int h,
int[] pixels) {
if (pixels == null || w == 0 || h == 0) {
return;
} else if (pixels.length < w * h) {
throw new IllegalArgumentException(
"pixels array must have a length >= w*h");
}
int imageType = img.getType();
if (imageType == BufferedImage.TYPE_INT_ARGB ||
imageType == BufferedImage.TYPE_INT_RGB) {
WritableRaster raster = img.getRaster();
raster.setDataElements(x, y, w, h, pixels);
} else {
img.setRGB(x, y, w, h, pixels, 0, w);
}
}

Reading and writing pixels from and to images would be completely useless if we did not process them in between operations. The implementation of the color tint equations is straightforward:

private void mixColor(int[] inPixels) {
int mix_a = mixColor.getAlpha();
int mix_r = mixColor.getRed();
int mix_b = mixColor.getBlue();
int mix_g = mixColor.getGreen();
for (int i = 0; i < inPixels.length; i++) {
int argb = inPixels[i];
int a = argb & 0xFF000000;
int r = (argb >> 16) & 0xFF;
int g = (argb >> 8) & 0xFF;
int b = (argb ) & 0xFF;
r = (int) (r * (1.0f - mixValue) + mix_r * mixValue);
g = (int) (g * (1.0f - mixValue) + mix_g * mixValue);
b = (int) (b * (1.0f - mixValue) + mix_b * mixValue);
inPixels[i] = a < <  24 | r < <  16 | g < <  8 | b;
}
}

Before applying the equations, we must split the pixels into their four color components.

Some bit shifting and masking is all you need in this situation. Once each color component has been filtered, the destination pixel is computed by packing the four modified color components into a single integer. Figure 2 shows a picture tinted with 50 percent red.

The above implementation works well but can be vastly improved performancewise.

The ColorTintFilter class in the CustomImageOp project online offers a better implementation that uses a few tricks to avoid doing all of the computations in the loop.