Matt RaibleMatt Raible is a Web Architecture Consultant specializing in open source frameworks.

10+ YEARS


Over 10 years ago, I wrote my first blog post. Since then, I've authored books, had kids, traveled the world, found Trish and blogged about it all.

What have I been working on at Taleo?

2011 has been a year of great clients for me. I started working with O.co and very much enjoyed my time there, especially on powder days in Utah. The people were great, the contract was great (no end date), but the work was not my forte. I was on a project to modularize the main shopping site's codebase, which involved mostly refactoring. By refactoring, I mean creating new Maven projects, modifying lots of pom.xml files and literally moving files from one directory to another. IntelliJ made this easy, the hard part was refactoring tests, moving from EasyMock to Mockito and splitting classes into interfaces and implementations where appropriate. As a developer who likes developing UIs and visually seeing my accomplishments, the project wasn't that exciting. However, I knew that it was strategically important to O.co, so I didn't complain much.

In mid-May, I received a LinkedIn message from the Director of Software Engineering at Taleo.

This is OB, I am the Director of Software Engineering at Taleo. We are the 2nd largest Software as a Service company. I am building a new specialist UI team that will take the product to the next level. I am looking for someone to lead this initiative. If you are interested to have a chat about it, please let me know.

At that time, I'd never heard of Taleo and quickly recommended they not hire me.

This probably isn't the best position for me. While I am a good leader, I'm not willing to relocate from Denver. I've found that leaders usually do best when face-to-face with their developers.

This conversation continued back-and-forth where I explained how I wasn't willing to go full-time and I didn't want to leave Overstock. In the end, OB was persistent and explained how the position would entail lots of UI work and wouldn't require me to travel much. Our negotiations trailed off in June and resumed in July after I returned from vacation in Montana. Shortly after, we met each other's expectations, agreed on a start date and I started working at Taleo in early September.

When I started, there were three features they wanted to add to to Taleo Business Edition: Profile Pictures, Talent Cards and Org Charts. They knew the schedule was tight (8 weeks), but I was confident I could make it happen. At first, I groaned at the fact that they were using Ant to build the project. Then I smiled when I learned they'd standardized on IntelliJ and set things up so you could do everything from the IDE. After using Maven for many years, this setup has actually become refreshing and I rarely have to restart or long for something like JRebel. Of course, a new kick-ass laptop and awesome IDE make it so I rarely wait for anything to happen.

To give you a taste of how I implemented each of these new features in 8 weeks, I've broken them into sections below.

Profile Pictures
Adding profile pictures was a pretty simple concept, one you see on my social networking sites today. I needed to give users the ability to upload a JPEG or PNG and crop it so it looked good. The uploading was a pretty straightforward process and I used a lot of internal APIs to grab the file from the request and save it to disk. The more difficult part was scaling the image to certain dimensions on upload (to save space) and allowing users to crop it after.

Most of Taleo Business Edition (TBE) is written in good ol' servlets and JSPs, with lots of scriptlets in their JSPs. When I saw the amount of HTML produced from Java, I laughed out loud and cringed. Soon after, I breathed a sigh of relief when I learned that any new features could be written using FreeMarker templates, which IntelliJ has excellent support for.

For image resizing on upload, I used Chris Campbell's Image.getScaledInstance() tutorial. For creating thumbnails, I used a combination of scaling, getSubimage() and the Java Image I/O API. I made sure to write to BufferedOutputStream for scalability. For cropping images client-side, I used jQuery UI's Dialog and Jcrop, the jQuery image cropping plugin. Below is a screenshot of what the cropping UI looks like:

Taleo's TBE: Profile Picture

Talent Cards
Talent Cards were a whole different beast. Not only did they need to display profile pictures, they also needed to contain contact information, work history and a number of other data points. Not just for employees, but for candidates as well. They also needed to be rendered with tabs at the bottom that allowed you to navigate between different data sections.

Taleo's TBE: Talent Card I'll admit, most of the hard work for this feature was done by the server-side developers (Harish and Vlad) on my team. Vlad built the tabbed interface and Harish built the administrative section that allows you to add/remove/sort fields, as well as show and hide certain tabs. I performed most of my magic with jQuery, its clueTip plugin and good ol' CSS. I was particularly thankful for CSS3 and its border-radius, box-shadow and Justin Maxwell's tutorial on CSS String Truncation with Ellipsis. I used DWR to fetch all the data from the server using Ajax.

Talent Cards are a slick feature in TBE 11.5 and I think they're a great way to see a lot of information about someone very quickly. If you enable them for your company, you'll be able to mouse over any employee or candidate's names and see their information.

Org Chart
The last feature I completed in this 8-week sprint was creating an organization chart. For this, I was given a rough prototype based on Caprica Software's JQuery/CSS Organisation Chart. When I received it, it had all kinds of cool CSS 3 transformations (like this one), but they only worked in Safari and Chrome. I ended up removing the transformations and adding the ability to navigate up and down the org tree with Ajax (we currently only show three levels at a time).

The Org Chart feature also allows you to see how many direct/indirect reports an employee has, as well as access their Talent Card by hovering over their name. It's one of my favorite features because it's so visual and because it builds upon all the other features we've built.

Taleo's TBE: Org Chart

Summary
As you might've guessed by now, I've been having a lot of fun doing UI development over the last few months. While I seem to have a knack for backend Java development, I enjoy developing UIs a lot more. The smile you see on people's faces during demos is priceless. I can't help but think this kind of thing contributes greatly to my developer happiness. All these features will be in next week's release of TBE and I couldn't be happier.

If you'd like to work on my team at Taleo (or even take over my current role as UI Architect), please drop me a line. If you live near their headquarters (Dublin, CA), it'd also be great to see you at the next Silicon Valley Spring User Group meetup. I'll be speaking about What's New in Spring 3.1 on February 1st.

Posted in Java at Dec 09 2011, 12:57:36 PM MST 1 Comment
Comments:

I don't know if this helps with your image stuff, but happened to have some random thing laying around which handles some fairly elaborate image related needs and do so very fast and handles almost anything you can throw at it.

This gets used to batch process a shit ton of images for people who really really care about image quality with a very elaborate set of methods for how images are converted to different formats / dimensions / cropping styles / etc.. It's been relentlessly performance profiled and tweaked to the point of whoever did it was some real sick puppy. Merry Christmas. =)

p.s. The recommended usage if you plan on performing more than one operation on a given image is to first call getbaselineimage and then pass that planarimage in for every operation you need.

p.p.s You can give it a tiff image but I wouldn't expect it to do what you want if it's a real tiff with multiple images inside of it. Think it does something random like use the first one.

(had to take some imports out to protect the innocent):

import com.google.common.base.Supplier;
import com.google.common.base.Suppliers;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import com.google.common.collect.Lists;
import com.google.common.io.Closeables;
import com.sun.media.jai.codec.FileSeekableStream;
import com.sun.media.jai.codec.JPEGEncodeParam;
import com.sun.media.jai.codec.SeekableStream;
import jaitools.tilecache.DiskMemTileCache;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.imageio.ImageIO;
import javax.inject.Inject;
import javax.media.jai.BorderExtender;
import javax.media.jai.BorderExtenderConstant;
import javax.media.jai.Interpolation;
import javax.media.jai.JAI;
import javax.media.jai.LookupTableJAI;
import javax.media.jai.PlanarImage;
import javax.media.jai.RenderedOp;
import javax.media.jai.operator.LookupDescriptor;
import javax.media.jai.operator.StreamDescriptor;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.Point;
import java.awt.RenderingHints;
import java.awt.geom.AffineTransform;
import java.awt.image.BufferedImage;
import java.awt.image.ColorConvertOp;
import java.awt.image.ColorModel;
import java.awt.image.Raster;
import java.awt.image.RenderedImage;
import java.awt.image.WritableRaster;
import java.awt.image.renderable.ParameterBlock;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.List;
import java.util.Set;

@Service
public class ImageServiceImpl implements ImageService
{
  private static final Logger log =
LoggerFactory.getLogger(ImageServiceImpl.class);

  private final Supplier<Set<ImageType>> supportedTypes =
Suppliers.memoize(new Supplier<Set<ImageType>>()
  {
     @Override
     public Set<ImageType> get()
     {
        List<ImageType> imgTypes = Lists.newLinkedList();
        ImageType[] types = ImageType.values();

        for (String format : ImageIO.getWriterFormatNames())
        {
           for (ImageType type : types)
           {
              if (type.getFormatName().equalsIgnoreCase(format))
              {
                 imgTypes.add(type);
                 break;
              }
           }
        }

        imgTypes.add(ImageType.TIFF);
        return ImmutableSet.copyOf(imgTypes);
     }
  });

  private final Supplier<RenderingHints> renderHints =
Suppliers.memoize(new Supplier<RenderingHints>()
  {
     @Override
     public RenderingHints get()
     {
        ImmutableMap.Builder<RenderingHints.Key,Object> bld =
ImmutableMap.builder();

        bld.put(JAI.KEY_BORDER_EXTENDER,
BorderExtender.createInstance(BorderExtenderConstant.BORDER_REFLECT));
        bld.put(RenderingHints.KEY_INTERPOLATION,
RenderingHints.VALUE_INTERPOLATION_BICUBIC);
        bld.put(RenderingHints.KEY_RENDERING,
RenderingHints.VALUE_RENDER_QUALITY);
        bld.put(RenderingHints.KEY_ALPHA_INTERPOLATION,
RenderingHints.VALUE_ALPHA_INTERPOLATION_QUALITY);
        bld.put(RenderingHints.KEY_FRACTIONALMETRICS,
RenderingHints.VALUE_FRACTIONALMETRICS_ON);
        bld.put(JAI.KEY_CACHED_TILE_RECYCLING_ENABLED, Boolean.TRUE);
        bld.put(JAI.KEY_INTERPOLATION,
Interpolation.getInstance(Interpolation.INTERP_BICUBIC_2));

        JAI defJai = JAI.getDefaultInstance();
        bld.put(JAI.KEY_TILE_CACHE, cache);

        RenderingHints rh = new RenderingHints(bld.build());
        defJai.setRenderingHints(rh);

        return rh;
     }
  });

  private final Supplier<JPEGEncodeParam> jpegEncodeParam =
Suppliers.memoize(new Supplier<JPEGEncodeParam>()
  {
     @Override
     public JPEGEncodeParam get()
     {
        JPEGEncodeParam param = new JPEGEncodeParam();
        param.setQuality(1.0f);

        return param;
     }
  });

  private FfmpegService ffmpegSvc;
  private final DiskMemTileCache cache;

  public ImageServiceImpl()
  {
     JAI defJai = JAI.getDefaultInstance();

     cache = new DiskMemTileCache();
     cache.setAutoFlushMemoryEnabled(true);
     cache.setMemoryCapacity(1024L * 1024L * 36);

     defJai.setTileCache(cache);
  }

  @Override
  public ImageFile getMetadata(File image)
     throws IOException
  {
     ImageType type =
ImageType.fromExtension(FileUtil.getExtension(image.getName()));

     if (type == ImageType.JPEG)
     {
        try
        {
           ImageFile imgMeta = JpegMetadataReader.parseImageMetaData(image);

           if (imgMeta.getDimension() != null)
              return imgMeta;
        } catch (Throwable t)
        {
           log.warn("Error parsing jpeg image meta for file: {}.
Falling back to basic jai dimension parsing.",
                    image, t);
        }
     }

     SeekableStream seekIs = null;
     try
     {
        seekIs = new FileSeekableStream(image);

        return parseMeta(seekIs);
     }  catch (Throwable t)
     {
        log.error("Unable to parse image with jai, either not a valid
image or format very exotic for file {}",
                  image, t);
        return null;
     } finally
     {
        Closeables.closeQuietly(seekIs);
     }
  }

  ImageFile parseMeta(SeekableStream stream)
  {
     RenderedOp srcOp = StreamDescriptor.create(stream, null,
renderHints.get());

     ImageFile ret = new JpegImageFile();
     ret.setDimension(new Dimension(srcOp.getWidth(), srcOp.getHeight()));

     srcOp.dispose();

     return ret;
  }

  @Override
  public ImageFile getMetadata(String fileName, InputStream source,
Object... args)
     throws IOException
  {
     ImageType type = ImageType.fromExtension(FileUtil.getExtension(fileName));

     boolean parseJpegMeta = true;
     if (args != null && args.length > 0)
        parseJpegMeta = (Boolean) args[0];

     if (parseJpegMeta && type == ImageType.JPEG)
     {
        ImageFile imgFile = JpegMetadataReader.parseImageMetaData(source);
        if (imgFile == null || imgFile.getDimension() == null)
        {
           if (source.markSupported())
              source.reset();
           else
              throw new RuntimeException("Unable to parse image
metadata from standard jpeg metadata reader:" + imgFile);
        }
     }

     SeekableStream seekIs = null;
     try
     {
        seekIs = SeekableStream.wrapInputStream(source, true);

        return parseMeta(seekIs);
     } finally
     {
        Closeables.closeQuietly(seekIs);
     }
  }

  @Override
  @Transactional(readOnly = true)
  public void convert(ImageType toType, InputStream is, OutputStream os)
     throws Exception
  {
     convertTo(toType, is, os);
  }

  @Override
  @Transactional(readOnly = true)
  public void convert(final ImageType toType, final PlanarImage
srcImage, final OutputStream os)
     throws Exception
  {
     convertTo(toType, srcImage, os);
  }

  @Override
  @Transactional(readOnly = true)
  public void convert(ImageType toType, InputStream is, OutputStream
os, float width, float height)
     throws Exception
  {
     convertTo(toType, is, os, width, height);
  }

  @Override
  @Transactional(readOnly = true)
  public void convert(ImageType toType, PlanarImage srcImage,
OutputStream os, float width, float height)
     throws Exception
  {
     convertTo(toType, srcImage, os, width, height);
  }

  @Override
  @Transactional(readOnly = true)
  public void convert(ImageType toType, InputStream is, OutputStream
os, double scaleFactor)
     throws Exception
  {
     convertTo(toType, is, os, scaleFactor);
  }

  static final byte[][] tableData = new byte[3][0x10000];
  static
  {
     double gamma = 1.0;
     double x;
     for (int i = 0; i < 256; i++)
     {
        x = i / 256.0;
        x = 255.0 * Math.pow(x, gamma);
        tableData[0][i] = (byte) x;
        tableData[1][i] = (byte) x;
        tableData[2][i] = (byte) x;
     }
  }

  @Override
  public PlanarImage getBaselineImage(final InputStream is)
     throws Exception
  {
     BufferedImage img = ImageIO.read(is);

     if (img != null && img.getType() != BufferedImage.TYPE_BYTE_INDEXED)
        return LookupDescriptor.create(img, new
LookupTableJAI(tableData), renderHints.get());

     PlanarImage srcOp = PlanarImage
        .wrapRenderedImage(img == null
                           ?
StreamDescriptor.create(SeekableStream.wrapInputStream(is, true),
null, renderHints.get())
                           : img);

     return srcOp.getColorModel().hasAlpha()
            ? convertTransparentImage(srcOp)
            : srcOp;
  }

  @Override
  @Transactional(readOnly = true)
  public void convert(ImageType toType, PlanarImage srcOp, OutputStream os,
                      float width, float height, Color background)
     throws Exception
  {
     RenderedImage scaleImage =
unsharpen(scaleImage(translate(srcOp), width, height),
                                          new
DimScale(srcOp.getWidth(), srcOp.getHeight(), width, height));

     if (scaleImage.getColorModel().hasAlpha())
     {
        int iwidth = Math.round(width);
        int iheight = Math.round(height);
        BufferedImage newImage = new BufferedImage(iwidth, iheight,
BufferedImage.TYPE_INT_RGB);
        Graphics2D g2 = (Graphics2D) newImage.getGraphics();
        g2.setRenderingHints(renderHints.get());
        g2.setColor(background);
        g2.fillRect(0, 0, iwidth, iheight);

        g2.drawRenderedImage(scaleImage,
AffineTransform.getScaleInstance(1.0, 1.0));

        JAI.create("encode", newImage, os, toType.getFormatName(),
                   toType == ImageType.JPEG ? jpegEncodeParam.get() : null);
        g2.dispose();
     } else
     {
        JAI.create("encode", scaleImage, os, toType.getFormatName(),
                   toType == ImageType.JPEG ? jpegEncodeParam.get() : null);
     }

     srcOp.dispose();
  }

  /**
   * If applicable to the given image, performs a jai <em>UnsharpMask</em>
   * operation on it to make smaller image appearance more pleasing
to human eye.
   *
   * @param srcOp   The image to potentially unsharpen.
   * @param dimScale   Optional description of total scale operation.
   * @return  The passed in image if it doesn't need unsharpening, else
   * the newly unsharpened image.
   *
   * @see <a href="http://download.oracle.com/docs/cd/E17802_01/products/products/java-media/jai/forDevelopers/jai-apidocs/javax/media/jai/operator/UnsharpMaskDescriptor.html">UnsharpMaskDescriptor</a>
   */
  RenderedImage unsharpen(RenderedImage srcOp, DimScale dimScale)
  {
     if (log.isDebugEnabled())
        log.debug("unsharpen() {}", dimScale);

     if (dimScale == null || dimScale.width > 0.9 || dimScale.height > 0.9)
        return srcOp;

     ParameterBlock pb = new ParameterBlock();
     pb.addSource(srcOp);
     pb.add(null); // convolution kernel

     // TODO: Replace hard coded gain which works well with images below
     // 110x110 dimensions to an agorithm that derives the ideal value
     // based on incoming image. If derived value does not fall in the
     // range of [-1,2] then image doesn't need unsharpen and above
     // check for 110x110 can be removed

     if (dimScale.width < 0.3 || dimScale.width >= 0.6)
        pb.add(0.17F);
     else
        pb.add(-0.17F);

//      pb.add(0.27F); 0.5f 1.0f -0.27f default value for op is 1.0f
     return JAI.create("UnsharpMask", pb, renderHints.get());
  }

  private PlanarImage modColorModel(PlanarImage img, int type)
  {
     BufferedImage dst = new BufferedImage(img.getWidth(),
                                           img.getHeight(),
                                           type);

     ColorConvertOp op = new ColorConvertOp(renderHints.get());
     ColorModel cm = img.getColorModel();

     final int xtiles = img.getNumXTiles();
     final int ytiles = img.getNumYTiles();
     for (int tileY = 0; tileY < xtiles; tileY++)
     {
        for (int tileX = 0; tileX < ytiles; tileX++)
        {
           Raster rast = img.getTile(tileX, tileY);

           WritableRaster tile;
           tile = rast instanceof WritableRaster
                  ? (WritableRaster) rast
                  :
Raster.createWritableRaster(rast.getSampleModel(),rast.getDataBuffer(),
                                                new
Point(rast.getMinX(), rast.getMinY()));

           //wrap the tile and color model in a buffered image
           BufferedImage tmp = new BufferedImage(cm,

tile.createWritableTranslatedChild(0, 0),
                                                 cm.isAlphaPremultiplied(),
                                                 null);

           op.filter(tmp, dst.getSubimage(tile.getMinX(),
                                          tile.getMinY(),
                                          tile.getWidth(),
                                          tile.getHeight()));
        }
     }

     return PlanarImage.wrapRenderedImage(dst);
  }

  @Override
  @Transactional(readOnly = true)
  public void scale(ImageType toType, PlanarImage srcData, OutputStream os,
                    float width, float height, Color letterboxColor,
Scale scale)
     throws Exception
  {
     final int srcWidth = srcData.getWidth();
     final int srcHeight = srcData.getHeight();

     final Dim srcDim = new Dim(srcWidth, srcHeight);
     final Dim toDim = new Dim(width, height);
     final Dim dim = calculateDim(srcDim, toDim, scale);

     switch (scale){
        case Crop:
           convert(toType, crop(srcData, srcDim, toDim), os,
toDim.width, toDim.height, letterboxColor);
           break;
        case Direct:
           convert(toType, srcData, os, toDim.width, toDim.height,
letterboxColor);
           break;
        case Varies:
           convert(toType, srcData, os, dim.width, dim.height, letterboxColor);
           break;
        case Letterbox:
           letterbox(srcData, srcDim, toDim, dim, letterboxColor, toType, os);
           return;
        default:
           throw new IllegalArgumentException("Unknown scale type
given: " + scale);
     }
  }

  private PlanarImage crop(PlanarImage srcOp, Dim srcDim, Dim toDim)
  {
     if (log.isDebugEnabled())
        log.debug("crop src dimensions: {}, target dimensions: {}",
srcDim, toDim);

     float width,height,x,y;

     //TODO: Below branching statements are begging to be condensed.
     // Find what can / how to condense them to something less
spagehetti looking
     if (srcDim.width > srcDim.height && toDim.height > toDim.width)
     {
        width = srcDim.height / toDim.ratio;
        height = srcDim.height;
        x = -((srcDim.width - width) / 2f);
        y = 0;
     } else if (srcDim.width > srcDim.height && toDim.width > toDim.height)
     {
        if (srcDim.ratio < toDim.ratio)
        {
           width = srcDim.width;
           height = srcDim.width / toDim.ratio;
           x = 0;
           y = -((srcDim.height - height) / 2f);
        } else
        {
           width = toDim.ratio * srcDim.height;
           height = srcDim.height;
           x = -((srcDim.width - width) / 2f);
           y = 0;
        }
     } else if (srcDim.width < srcDim.height && toDim.width > toDim.height)
     {
        width = srcDim.width;
        height = srcDim.width / toDim.ratio;
        x = 0;
        y = -((srcDim.height - height) / 2f);
     } else if (srcDim.width < srcDim.height && toDim.width < toDim.height)
     {
        if (srcDim.ratio < toDim.ratio)
        {
           width = srcDim.height / toDim.ratio;
           height = srcDim.height ;
           x = -((srcDim.width - width) / 2f);
           y = 0;
        } else
        {
           width = srcDim.width;
           height = srcDim.width * toDim.ratio;
           x = 0;
           y = -((srcDim.height - height) / 2f);
        }
     } else if (srcDim.width == srcDim.height)
     {
        if (toDim.width == toDim.height)
           return translate(srcOp);

        if (toDim.width > toDim.height)
        {
           width = srcDim.width;
           height = srcDim.width / toDim.ratio;
           x = 0;
           y = -((srcDim.height - height) / 2f);
        } else
        {
           width = srcDim.height / toDim.ratio;
           height = srcDim.height ;
           x = -((srcDim.width - width) / 2f);
           y = 0;
        }
     } else
        return translate(srcOp);

     if (log.isDebugEnabled())
     {
        log.debug("crop() from src {} and target {} \n calculated new dim: {}",
                  new Object[]{srcDim, toDim, new Dim(width, height, x, y)});
     }

     ParameterBlock pb2 = new ParameterBlock();
     pb2.addSource(srcOp);
     pb2.add(x);
     pb2.add(y);

     srcOp = JAI.create("translate", pb2, renderHints.get());

     ParameterBlock pb = new ParameterBlock();
     pb.addSource(srcOp);
     pb.add(0f);
     pb.add(0f);
     pb.add(width);
     pb.add(height);

     return translate(JAI.create("crop", pb, renderHints.get()));
  }

  private PlanarImage letterbox(PlanarImage srcOp, Dim src, Dim to,
Dim ret, Color letterboxColor,
                                ImageType toType, OutputStream os)
     throws Exception
  {
     if (log.isDebugEnabled())
        log.debug("letterbox() src dim: {}, target dim: {}", src, to);

     // move x,y coordinates by whatever ret has first
     srcOp = translate(srcOp, ret);

     RenderedOp newOp = (RenderedOp) unsharpen(scaleImage(srcOp,
ret.width, ret.height),
                                               new
DimScale(src.width, src.height, ret.width, ret.height));

     BufferedImage newImage = new BufferedImage(Math.round(to.width),
Math.round(to.height), BufferedImage.TYPE_INT_RGB);

     Graphics2D g2 = (Graphics2D) newImage.getGraphics();
     g2.setRenderingHints(renderHints.get());
     g2.setColor(letterboxColor);
     g2.fillRect(0, 0, newImage.getWidth(), newImage.getHeight());

     g2.drawImage(newOp.getAsBufferedImage(), Math.round(ret.x),
Math.round(ret.y),
                  Math.round(ret.width), Math.round(ret.height), null);
     g2.dispose();

     return JAI.create("encode", newImage, os, toType.getFormatName(),
                       toType == ImageType.JPEG ?
jpegEncodeParam.get() : null);
  }

  PlanarImage translate(PlanarImage srcImg)
  {
     ParameterBlock pb = new ParameterBlock();
     pb.addSource(srcImg);
     pb.add(0f);
     pb.add(0f);

     return JAI.create("translate", pb, renderHints.get());
  }

  PlanarImage translate(PlanarImage srcOp, Dim dim)
  {
     if (dim.x == 0 && dim.y == 0)
        return srcOp;

     ParameterBlock pb = new ParameterBlock();
     pb.addSource(srcOp);
     pb.add(dim.x);
     pb.add(dim.y);

     return JAI.create("translate", pb, renderHints.get());
  }

  private void convertTo(ImageType toType, InputStream is,
OutputStream os, Object... args)
     throws Exception
  {
     if (!isSupported(toType))
        throw error("image.conversion.type.not-supported", toType);

     PlanarImage srcImage = getBaselineImage(is);

     if (args != null && args.length == 2)
        convert(toType, srcImage, os, ((Number)
args[0]).floatValue(), ((Number) args[1]).floatValue(), Color.WHITE);
     else
        convertTo(toType, srcImage, os, args);

     srcImage.dispose();
  }

  private void convertTo(ImageType toType, RenderedImage srcOp,
OutputStream os, Object... args)
     throws Exception
  {
     if (!isSupported(toType))
        throw error("image.conversion.type.not-supported", toType);

     if (args != null && args.length > 0)
        srcOp = unsharpen(scaleImage(srcOp, args), null);

     JAI.create("encode", srcOp, os, toType.getFormatName(),
                toType == ImageType.JPEG ? jpegEncodeParam.get() : null);
  }

  public RenderedImage scaleImage(RenderedImage srcOp, Object... args)
  {
     ParameterBlock pb = new ParameterBlock();
     pb.addSource(srcOp);

     if (args != null && args.length == 2)
     {
        final double width = ((Number) args[0]).doubleValue();
        final double height = ((Number) args[1]).doubleValue();

        double scaleWidth = width / srcOp.getWidth();
        double scaleHeight = height / srcOp.getHeight();

        if (scaleWidth == 1.0 && scaleHeight == 1.0)
           return srcOp;

        if (scaleWidth > 0.4 || scaleHeight > 0.4)
        {
           pb.add((float) scaleWidth);
           pb.add((float) scaleHeight);
           pb.add(0.0F);
           pb.add(0.0F);
           pb.add(Interpolation.getInstance(width < 100 ?
Interpolation.INTERP_BILINEAR : Interpolation.INTERP_BICUBIC_2));

           return JAI.create("scale", pb, renderHints.get());
        }

        pb.add(scaleWidth);
        pb.add(scaleHeight);
        pb.add(Interpolation.getInstance(Interpolation.INTERP_BICUBIC_2));

        return scaleImage(JAI.create("SubsampleAverage", pb,
renderHints.get()), width, height);

     } else if (args != null && args.length == 1)
     {
        final Double scale = (Double) args[0];

        pb.add(scale.floatValue());
        pb.add(scale.floatValue());

        return JAI.create("scale", pb, renderHints.get());
     } else
        throw new IllegalStateException("Unknown arguments given");
  }

  private PlanarImage convertTransparentImage(PlanarImage srcImage)
     throws Exception
  {
     return modColorModel(srcImage, BufferedImage.TYPE_4BYTE_ABGR);
  }

  static Dim calculateDim(Dim srcDim, Dim to, Scale scale)
  {
     Dim ret = new Dim(0, 0);

     switch (scale){
        case Letterbox:
           if (srcDim.width > srcDim.height || srcDim.width == srcDim.height)
              scaleWidth(ret, srcDim, to, scale);
           else
              scaleHeight(ret, srcDim, to, scale);

           ret.y = to.height > 0 ? (to.height - ret.height) / 2f : 0f;
           ret.x = to.width > 0 ? (to.width - ret.width) / 2f : 0f;
           /*
           if (ret.width / to.width >= 0.98f)
              ret.width = to.width;
           if (ret.height / to.height >= 0.98f)
              ret.height = to.height;*/

           return ret;
        case Crop:
        case Direct:
           ret.width = to.width;
           ret.height = to.height;
           return ret;
        case Varies:
           if (to.width > to.height || to.width == to.height)
              scaleWidth(ret, srcDim, to, scale);
           else
              scaleHeight(ret, srcDim, to, scale);
           return ret;
        default:
           throw new IllegalArgumentException("Unknown scale type: " + scale);
     }
  }

  static void scaleWidth(Dim ret, Dim srcDim, Dim to, Scale scale)
  {
     ret.width = to.width;
     ret.height = to.width / srcDim.ratio;

     // if overscaled,  scale back width by difference and re-calculate

     if (scale == Scale.Letterbox && ret.height > to.height)
     {
        float overSizedRatio = to.height / ret.height;
        ret.width *= overSizedRatio;
        ret.height = Math.round(ret.width / srcDim.ratio);
     } else if (scale == Scale.Varies && to.height == 0)
     {
        if (srcDim.width >= srcDim.height)
           ret.height = to.width / srcDim.ratio;
        else
           ret.height = to.width * srcDim.ratio;
     }

     if (log.isDebugEnabled())
        log.debug("scaleWidth() srcDim: {}\ntoDim: {}\nret: {}", new
Object[]{srcDim, to, ret});
  }

  static void scaleHeight(Dim ret, Dim srcDim, Dim to, Scale scale)
  {
     ret.width = to.height / srcDim.ratio;
     ret.height = to.height;

     // if overscaled,  scale back height by difference and re-calculate

     if (scale == Scale.Letterbox && ret.width > to.width)
     {
        float overSizedRatio = to.width / ret.width;
        ret.height *= overSizedRatio;
        ret.width = ret.height / srcDim.ratio;
     } else if (scale == Scale.Varies && to.width == 0)
     {
        if (srcDim.width >= srcDim.height)
           ret.width = to.height * srcDim.ratio;
        else
           ret.width = to.height / srcDim.ratio;
     }

     if (log.isDebugEnabled())
        log.debug("scaleHeight() srcDim: {}\ntoDim: {}\nret: {}", new
Object[]{srcDim, to, ret});

  }

  @Override
  public boolean isSupported(ImageType type)
  {
     return supportedTypes.get().contains(type);
  }

  @Override
  @Transactional(readOnly = true)
  public File captureVideoFrame(final VideoAssetFile file, final long
milliseconds)
  {
     FileInputStream is = null;
     FileOutputStream fo = null;
     File still = null;
     try
     {
        still = ffmpegSvc.captureVideoFrame(file, milliseconds);
        is = new FileInputStream(still);

        PlanarImage imgOp = getBaselineImage(is);

        File tmpJpg =
FileUtil.createTempFile(FileUtil.getBaseName(still.getName()),
".jpg");
        tmpJpg.deleteOnExit();

        fo = new FileOutputStream(tmpJpg);

        convert(ImageType.JPEG, imgOp, fo, imgOp.getWidth(),
imgOp.getHeight(), Color.WHITE);

        return tmpJpg;
     } catch (Throwable t)
     {
        log.error("Unable to capture frame for {}", file, t);
        return null;
     } finally
     {
        Closeables.closeQuietly(is);
        Closeables.closeQuietly(fo);

        if (still != null && !still.delete())
           log.warn("Unable to delete tmp still image {}", still);
     }
  }

  @Inject
  public void setFfmpegSvc(FfmpegService ffmpegSvc)
  {
     this.ffmpegSvc = ffmpegSvc;
  }

  static class DimScale {
     public double width;
     public double height;

     DimScale(float srcWidth, float srcHeight, float toWidth, float toHeight)
     {
        this.width = toWidth / srcWidth;
        this.height = toHeight / srcHeight;
     }

     DimScale(double width, double height)
     {
        this.width = width;
        this.height = height;
     }

     @Override
     public String toString()
     {
        final StringBuilder sb = new StringBuilder(80);
        sb.append("DimScale");
        sb.append("{width=").append(width);
        sb.append(", height=").append(height);
        sb.append('}');
        return sb.toString();
     }
  }

  static class Dim {
     public float width;
     public float height;
     public float x;
     public float y;
     public float ratio;

     Dim(float width, float height)
     {
        this.width = width;
        this.height = height;
        calcRatio();
     }

     Dim(float width, float height, float x, float y)
     {
        this(width, height);
        this.x = x;
        this.y = y;
     }

     public void calcRatio()
     {
        this.ratio = width > height ? width / height : height / width;
     }

     @Override
     public String toString()
     {
        final StringBuilder sb = new StringBuilder(40);
        sb.append("Dim");
        sb.append("{width=").append(width);
        sb.append(", height=").append(height);
        sb.append(", x=").append(x);
        sb.append(", y=").append(y);
        sb.append(", ratio=").append(ratio);
        sb.append('}');
        return sb.toString();
     }
  }
}

Posted by Bob on December 09, 2011 at 04:42 PM MST #

Post a Comment:
  • HTML Syntax: Allowed