Responsive images on the web

Abstract

After I've redesigned my home page and relocated it to the new domain, I encountered the problem to display the same image at different screen resolutions. Websites with inappropriate representation of images (mostly on mobile) force a user to close them before the browser even finished loading the remaining content. They let one wait longer, pay for additional traffic and they destroy the website's design. Nowadays internet users consume information on multiple devices with different screen resolutions, so they attach a great importance to the ability to surf a web page equally good on different screen sizes.

Objective

Implement a solution to display images at different screen resolutions, which works in browsers supported by google and requires a low pre-production effort.

Requirements

• HTML markup support for image source declaration (no CSS)
• Browser support for Chrome > 23, Firefox > 23, IE > 9, Safari > 6
• No additional traffic caused by image download, e.g. download a large image and let the browser resize it
• Low manual pre-processing effort, e.g. manual image resizing for different screen resolutions

Alternative evaluation

The process of displaying images at different screen resolutions can be broken down to the following two steps:

• to deliver a suitable image for the client's screen resolution and
• to prepare the image to be supplied in previous step.

Delivery

The delivery step is responsible for forcing the browser to download an image suitable to its screen resolution.

CSS Media Queries

The first solution comes to mind is simply to get rid of css media queries:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

@media (min-width: 601px) {
    #image {
        background: url('images/white.png') repeat-x;
    }
}

@media (max-width: 600px) {
    #image {
        background: url('images/white-mobile.png') repeat-x;
    }
}

Despite the approach is the most robust and proven, it breaks the first requirement.

<picture> tag

The HTML <picture> element is a container used to specify multiple <source>'s for an image. The browser will choose the most suitable source according to the current layout of the page and the screen it will be displayed on:

1
2
3
4
5
6

<picture>
    <source srcset="pear-mobile.jpeg" media="(max-width: 720px)">
    <source srcset="pear-tablet.jpeg" media="(max-width: 1280px)">
    <source srcset="pear-desktop.jpeg">
    <img src="pear-tablet.jpeg" alt="The pear is juicy.">
</picture>

<picture> was introduced in the HTML 5.1 draft and implemented only in the last versions of the browsers, e.g. Firefox > 38, Chrome > 43. The lack of IE and Safari support makes it impossible to use the plain <picture> element. Fortunately, there is a plenty of polyfills which enables <picture> element in browsers without its native support.

Preparation

The preparation step is responsible for scaling images to be delivered to the device browser to its screen resolutions.

Content-aware image cropping

The content-aware image cropping is a technique of discovering and cutting out the point of interest of the image content. One possible and most widely used application of this method is a thumbnail-cropping. The simplest way to create a thumbnail is to proportionally resize the original image to fit a width or height. However, if both dimensions are constrained, you usually have to crop the image before resizing.¹ The most implementations detect edges using laplace and use additional methods to find important image regions, e.g. regions high in saturation or with a color like skin.² They produce relatively good results on images with one clearly distinguishable point of interest:

Source image

Point of interest

and as expected fail to choose the right one (from human perspective) on image with several possible:

Source image

Point of interest

The best implementation I've found tries to find the best cropping position of the window that maximizes the number of point of interests it contains.

Content-aware image scaling

The content-aware image scaling or seam carving use not only geometric constraints, but consider the image content as well. It supports content-aware image resizing for both reduction and expansion. A seam is an optimal 8-connected path of pixels on a single image from top to bottom, or left to right, where optimality is defined by an image energy function. The aspect ratio of an image can be changed by repeatedly carving out or inserting seams in one direction.⁴ ImageMagick provides a support for seam carving with -liquid-rescale option:

Before

After

Manual image scaling

The manual image scaling is the most time-consuming but also the most qualitative resizing approach, because the point of interest discovery is entirely under human control.

Conslusion

The media queries are violating the first requirement about explicit declaration of the image source in the HTML markup, which is supported by the <picture> tag. Both approaches were unable to meet the last requirement and require the same huge pre-production effort.

The pre-production effort could be reduced by an application, which resizes or crops the main image without interaction with a human. The drawback of using such an application without any advice is the lost of predictability and often ugly or indistinguishable looking images.

Hereby the <picture> tag with the manual image scaling are the most valuable solutions for specified requirements.

Implementation

To be able to use the <picture> tag in my pelican blog I've implemented the mdx_picture markdown extension, which is able to generate valid HTML:

1
2
3
4
5

<picture>
    <source srcset="pear-tablet.jpeg" media="(max-width: 1280px)">
    <source srcset="pear-mobile.jpeg" media="(max-width: 720px)">
    <img alt="This picture loads on non-supporting browsers." src="image.jpg" title="The image title" />
</picture>

from the following markdown:

1
2
3
4
5

[picture]
    [720px]: pear-mobile.jpeg
    [1280px]: pear-tablet.jpeg
    ![This picture loads on non-supporting browsers.](image.jpg "The image title")
[/picture]

The generated HTML works out of the box in all browsers with <picture> tag support. The rest gets enabled by the picturefill.js polyfill.