How to build an Apple-style scroll image sequence animation

Why stills on a canvas, not a video

The obvious way to tie footage to the scroll is a <video> whose currentTime follows the scroll. Every scroll change is then a seek, and a video can only seek by going back to the keyframe before the frame you asked for and decoding forward from there. Encoders put keyframes seconds apart, because that is cheap for playback: left at their defaults, VP9 and H.264 both put only 2 keyframes in this site’s 5.4-second, 162-frame film. Timed on the scroll video scrub generator, the slowest 5% of seeks on those files took 24–116 ms across Chromium, Firefox and WebKit, when a frame at 60 fps lasts 16.7 ms. That is the stutter.

An image sequence has no keyframes. Each frame is its own picture, fetched and decoded once, so any scroll position can show its frame at once, in every browser: drawing a decoded image onto a canvas is a copy. The scroll names a frame number and that exact picture is drawn, with no codec, no Range requests and no seek in flight to wait for.

Turn a video into an image sequence

Most sequences start as a video: a 3D render, a turntable shot, a screen recording. The scroll image sequence generator does it in the browser: pick a file, a frame count, a width and WebP or JPEG, and it seeks the video to the middle of each equal slice in turn, draws that picture onto a canvas and saves it. Download frames.zip hands you frame_0001.webp and on, and nothing is uploaded. Safari’s canvas cannot write WebP, so there it writes JPEG.

From a terminal, for a long video or a build step, ffmpeg does the same:

mkdir frames

# 12 frames for each second of video, 960px wide (the height follows,
# rounded to an even number), as WebP: frames/frame_0001.webp …
ffmpeg -i input.mp4 -an -vf "fps=12,scale=960:-2" \
  -c:v libwebp -quality 80 frames/frame_%04d.webp

# How many came out: the COUNT to use
ls frames | wc -l

Run with ffmpeg 9.0 on this site’s 5.4 s film, fps=12 gives 65 frames, 960×720, numbered from frame_0001. The libwebp line needs an ffmpeg built with it, as most are; for JPEG, swap the codec options for -q:v 3 and the extension for .jpg.

Draw it by hand: a sticky stage and a canvas

The page needs a tall track that holds the scroll room and a stage inside it that sticks to the top of the screen while the track goes by. The canvas fills the stage. With a 400vh track, the stage stays stuck for 300vh of scroll, and that is how long the film runs.

<section class="track" id="track">
  <div class="stage">
    <canvas id="film" role="img"
      aria-label="Landsat 8 turning a full circle as you scroll"></canvas>
  </div>
</section>

/* The track holds the scroll room: the stage stays stuck for the
   track's height minus one screen, and the film runs that long. */
.track { height: 400vh; }
.stage { position: sticky; top: 0; height: 100vh; }
.stage canvas { display: block; width: 100%; height: 100%; }

The script loads the frames, works out which one the scroll calls for, and draws it:

const COUNT = 71;
const url = (i) => `frames/frame_${String(i + 1).padStart(4, '0')}.webp`;

const track = document.getElementById('track');
const canvas = document.getElementById('film');
const ctx = canvas.getContext('2d');
const reduced = matchMedia('(prefers-reduced-motion: reduce)').matches;

const images = [];                     // images[i] once frame i has loaded
let wanted = reduced ? COUNT - 1 : 0;  // the frame the scroll calls for
let drawn = -1;                        // the frame on the canvas

// Load every frame; under reduced motion, only the last one.
for (let i = reduced ? COUNT - 1 : 0; i < COUNT; i++) {
  const img = new Image();
  img.onload = () => { images[i] = img; draw(); };
  img.src = url(i);
}

// The loaded frame nearest i, so the canvas never waits for one.
function nearest(i) {
  for (let d = 0; d < COUNT; d++) {
    if (images[i - d]) return i - d;
    if (images[i + d]) return i + d;
  }
  return -1;
}

function draw() {
  const i = nearest(wanted);
  if (i < 0 || i === drawn) return; // nothing new to show
  const img = images[i];
  const { width: cw, height: ch } = canvas;
  // Cover: scale until the box is full, centre, crop what spills over.
  const s = Math.max(cw / img.naturalWidth, ch / img.naturalHeight);
  const w = img.naturalWidth * s;
  const h = img.naturalHeight * s;
  ctx.clearRect(0, 0, cw, ch);
  ctx.drawImage(img, (cw - w) / 2, (ch - h) / 2, w, h);
  drawn = i;
}

function update() {
  if (!reduced) {
    // 0 when the track's top reaches the top of the screen, 1 when
    // its bottom reaches the bottom: exactly while the stage is stuck.
    const r = track.getBoundingClientRect();
    const p = Math.min(1, Math.max(0, -r.top / (r.height - innerHeight)));
    wanted = Math.round(p * (COUNT - 1));
  }
  draw();
}

// The drawing buffer is the CSS box × devicePixelRatio (2 at most),
// so the picture is sharp on a high-density screen.
function resize() {
  const dpr = Math.min(devicePixelRatio || 1, 2);
  canvas.width = Math.round(canvas.clientWidth * dpr); // this clears it
  canvas.height = Math.round(canvas.clientHeight * dpr);
  drawn = -1;
  update();
}

// One update per animation frame at most; passive, so it never
// holds up the scroll.
let queued = false;
if (!reduced) {
  addEventListener('scroll', () => {
    if (queued) return;
    queued = true;
    requestAnimationFrame(() => { queued = false; update(); });
  }, { passive: true });
}
addEventListener('resize', resize);
resize();

Five details carry it. The scroll listener is passive and only queues one requestAnimationFrame, so the work runs once per frame, however many scroll events arrive. Progress is the track’s top scrolled past, over the track’s height minus one screen, so it runs from 0 to 1 exactly while the stage is stuck. The canvas’s drawing buffer is its CSS size times devicePixelRatio, capped at 2, or the picture is soft on a phone; setting it clears the canvas, so resize redraws. The cover arithmetic scales the frame by the larger of the two ratios and centres it, so the box is always full. And nearest means a frame that has not arrived yet never leaves the canvas blank: the closest one that has is drawn, and draw does nothing when the frame to show has not changed.

The same with scrollFrames

svg-scroll-draw’s scrollFrames, from its own svg-scroll-draw/frames entry (4.4 KB), does the script above in one call, on the same HTML and CSS:

import { scrollFrames } from 'svg-scroll-draw/frames';

// The same track, stage and canvas as above.
scrollFrames('#film', {
  frames: (i) => `frames/frame_${String(i + 1).padStart(4, '0')}.webp`,
  count: 71,
  // The canvas is in a sticky stage, so its own box moves with the
  // scroll: measure the track that holds the scroll room instead.
  triggerElement: document.getElementById('track'),
  trigger: { start: 'top top', end: 'bottom bottom' },
});

It draws frame round(progress × (count − 1)), or the nearest loaded one until that arrives, and redraws only when the frame to show changes. What it adds is the loading. It fetches frame 0 first, then the last, then the middle of every gap, coarse to fine (Fig. 1), at most 6 at a time (concurrency), so a reader who scrolls fast is never far from a real frame after a handful of loads. A frame that fails to load is skipped for its neighbour, with one console warning naming its URL. It resizes the drawing buffer with a ResizeObserver, not only on window resizes, and destroy() stops the loads still in flight.

triggerElement is what makes the pinned version work: the canvas sits in a sticky stage, so its own box moves with the scroll for as long as it is stuck, and a window measured on it comes out wrong. Measure the track with 'top top' to 'bottom bottom' instead, as above. Leave triggerElement out for a canvas that scrolls with the page, and the window defaults to 'top top' → 'bottom top': the film runs as the canvas leaves. fit: 'contain' shows the whole frame instead of cropping, and onProgress(alpha, frame) hands you the frame number for a counter or captions. Every option is in the scrollFrames docs.

The scroll image sequence example is this listing on 71 frames of NASA’s Landsat 8, 960×540, with chapters of copy that follow the frame, and Groundtrack is a whole site built around it.

Budget the frames

A sequence is paid for frame by frame. On the wire, WebP keeps it small: the example’s 71 frames are 767 KB together. In memory it is another matter, because a decoded frame is its width × height × 4 bytes, and a sequence keeps every frame it has loaded so it can show any of them at once. One 960×540 frame is 2.1 MB decoded; all 71 are 147 MB, about 190 times their size on disk. A hundred 1920×1080 frames would be 829 MB.

• Make each frame the size it is shown. The canvas never draws more than its box × devicePixelRatio, capped at 2, so a frame wider than twice the canvas’s widest box costs bytes and memory for nothing. Export WebP, or JPEG, at that size, not the camera’s originals.
• Use as few frames as the motion needs. The example spreads 71 frames over 300vh of scroll, about 4.3vh each. Halving the count halves the bytes and the memory; the generator shows the scroll each frame covers and the weight of the set as you change them.
• Cache them for good. The frames do not change once made: give the folder a long cache lifetime and rename it when they do.

Accessibility and reduced motion

A canvas is a blank box to a screen reader. Give it role="img" and an aria-label that says what the film shows, as the HTML above does: scrollFrames adds no ARIA, so that is up to you. Any copy that changes with the frame should be real text beside the canvas, not painted into the frames.

A film scrubbed by the scroll is motion the reader did not start, so under prefers-reduced-motion: reduce both versions load only the last frame, draw it and leave the scroll alone. scrollFrames also stops on the last frame if the setting is turned on during the visit, and loads nothing more. Make that last frame one that stands on its own, and if chapters of copy follow the film, show them all, as the example does. What every other kind of scroll effect should do is in reduced motion for scroll animations.