Skip to content

Latest commit

 

History

History
71 lines (54 loc) · 2.57 KB

README.md

File metadata and controls

71 lines (54 loc) · 2.57 KB
Raw

Responsive Image Container

This repo specifies and prototypes a new responsive image container.

The basic principles:

  • Support image data in multiple formats, and especially WebP & JPEG-XR.
  • Image data will be represented in layers, each layer representing a certain resolution.
  • The first layer represents the lowest resolution image, and each layer above it represents the difference between its resolution and the resolution below it.
  • The container will define a set of algorithms that can be used to define the diff calculation between the layers

Algorithms:

  • Simple diff - something like this proposal can work great for resolution switching
  • Maybe can be extended with better extrapolation algorithms, such as this one
  • Placing the lower layer as an image fragment in the layer above it - Can be used for art-direction

Container format:

After hesitating between RIFF, Matroska and ISO base media format, I've decided to go with the ISO media format, because of its simplicity, extensibility and the fact that it is streaming friendly, (the entire file's size is not required before writing the file's header).

How will browsers fetch that format?

Wrote this blog post a while back.

Downsides of this solution

  • Decoding of this container will consume more than traditional sequantial JPEGs, since it'd require layer upscaling.
  • This may be eased by using the GPU for upscaling.
  • Network performance in HTTP/1.1 may be an issue. Needs to be investigated.
  • Monochrome or print specific images are not supported.
  • If art-directed images of lower resolution are completely different than their higher resolution counterparts, this solution is no good for that case.
  • A separate resource would be better there.
  • There seems to be a consensus that this case is rather rare.

Dependencies

  • libjpeg
  • libzlib
  • libwebp
  • PIL

On ubuntu you can run sudo apt-get install libjpeg-dev zlib1g-dev libwebp-dev python-pillow

Usage

Encoding: ./ric_encoder.py encode <input> <output> <config> Decoding: ./ric_encoder.py decode <input> <output>

The config files are json files that indicate the output width and crop for each layer. Examples can be found in samples/*_config.txt.