Measuring scroll FPS via headless Chrome

As part of my day job, I’ve been tackling scrolling performance in our main editor (which mixes text content with large tables of data). Most web performance tooling targets page loading times: Time to First Byte, First Contentful Paint, etc. Important metrics for sure, especially for content-centric pages, but not the focus of our long-running web app. Scrolling is one of the most common interactions for a document app, and there aren’t any off-the-shelf automated tools that give you a consistent way to measure it.

I spent some time investigating, and there’s surprisingly little out there. I didn’t come up with anything particularly earth-shattering here, but figured I’d share for the next unfortunate soul that has to go down the same path.

But why not just measure manually?

Sure. You can open DevTools, hit the Performance tab, scroll around, and look at the frames chart (or turn on the FPS meter in the Rendering tab). Fine as a one-off, but it’s not at all repeatable: different machines give different numbers, DevTools adds overhead, doesn’t run in CI, and so on. To track regressions, you need something programmatic that gives consistent results.

Enter Puppeteer

Puppeteer gives us headless Chrome with a nice Node.js API. A naive approach using window.scrollBy() technically works, but doesn’t at all match what happens for a user with a mouse. Real scroll gestures go through the browser’s input handling pipeline, which is what we care about for measuring jank.

Chrome DevTools Protocol (CDP) has Input.synthesizeScrollGesture (marked experimental, but it works fine) for exactly this type of scenario:

const client = await page.target().createCDPSession();

await client.send('Input.synthesizeScrollGesture', {
  x: 640,
  y: 400,
  yDistance: -3000, // negative = scroll down
  speed: 800, // pixels per second
  repeatCount: 2, // repeats, so 3 gestures total
  repeatDelayMs: 500,
});

This is much closer to what happens with a real wheel or trackpad. Multiple scroll events are fired and frames are rendered while scrolling, unlike window.scrollBy() which just jumps straight to the target.

Take 1: Counting frames via requestAnimationFrame

The simplest possible thing is to count frames using requestAnimationFrame. Using page.evaluate(), we inject some counting code into the page itself:

// Don't await here, needs to run *during* the scroll gesture
const fpsPromise = page.evaluate(() => {
  window.scrollDone = false;

  return new Promise(resolve => {
    const start = performance.now();
    let frames = 0;

    function tick() {
      frames++;

      if (window.scrollDone) {
        const seconds = (performance.now() - start) / 1000;
        resolve(frames / seconds);
        return;
      }

      requestAnimationFrame(tick);
    }

    requestAnimationFrame(tick);
  });
});

await client.send('Input.synthesizeScrollGesture', {
  x: 640,
  y: 400,
  yDistance: -3000,
});
await page.evaluate(() => {
  window.scrollDone = true;
});

const fps = await fpsPromise;

This is okay-ish. We’re only measuring the main thread. If your scroll jank comes from JS on the main thread, requestAnimationFrame might be all you need. Chrome scrolls on the compositor thread whenever it can, blocking event listeners (e.g. wheel) force the browser to wait for the handler before rendering (wheel listeners on window, document, and body are passive by default as of Chrome 73, which helps). When the compositor does handle scrolling you can still get issues: a busy main thread can drop frames while the compositor stays smooth, steady 60fps from requestAnimationFrame can hide the compositor dropping frames.

Averages here can be deceiving: if you have a 400ms frame during an otherwise smooth scroll, your numbers won’t look terrible but users will definitely notice.

Take 2: CDP tracing

For a more robust measurement, you can use CDP’s tracing via Puppeteer’s page.tracing, which captures events from both the main and compositor threads.

await page.tracing.start({ path: 'trace.json', screenshots: false });

// ... do your scrolling here ...

await page.tracing.stop();

This creates a trace.json with timestamped events, including one for every frame drawn. Filter to DrawFrame events and calculate the frame rate:

const fs = require('fs');
const trace = JSON.parse(fs.readFileSync('trace.json', 'utf8'));

const frames = trace.traceEvents.filter(e => e.name === 'DrawFrame');

// Sort by timestamp and compute intervals
const timestamps = frames.map(f => f.ts).sort((a, b) => a - b);
const intervals = timestamps.slice(1).map((ts, i) => ts - timestamps[i]);
const avgInterval = intervals.reduce((a, b) => a + b, 0) / intervals.length;
const fps = 1e6 / avgInterval; // timestamps are in microseconds

This gives you better information, but also has issues:

Also, the long-frame problem from earlier still applies: track long frames and surface them separately.

Some of this is left as an exercise to the reader.

Putting it all together, warts and all:

// Usage: node scroll-fps.js https://example.com
const puppeteer = require('puppeteer');

const FRAME_BUDGET_US = 1e6 / 60; // ~16.7ms, in microseconds

(async () => {
  const url = process.argv[2];
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.setViewport({ width: 1280, height: 800 });
  await page.goto(url, { waitUntil: 'networkidle0' });

  const client = await page.target().createCDPSession();

  await page.tracing.start({ screenshots: false });
  await client.send('Input.synthesizeScrollGesture', {
    x: 640,
    y: 400,
    yDistance: -3000, // negative = scroll down
    speed: 800, // pixels per second
  });
  const trace = JSON.parse(await page.tracing.stop());

  await browser.close();

  // Over-simplified: Should still filter to the scroll window
  // (via performance.mark), to a single pid, etc
  const timestamps = trace.traceEvents
    .filter(e => e.name === 'DrawFrame')
    .map(e => e.ts)
    .sort((a, b) => a - b);
  const intervals = timestamps.slice(1).map((ts, i) => ts - timestamps[i]);
  const avgInterval = intervals.reduce((a, b) => a + b, 0) / intervals.length;
  const slowFrames = intervals.filter(i => i > FRAME_BUDGET_US * 1.5).length;

  console.log(`Frames: ${timestamps.length}`);
  console.log(`Avg FPS: ${(1e6 / avgInterval)}`);
  console.log(
    `Slow frames: ${slowFrames} (${((slowFrames / intervals.length) * 100)}%)`
  );
})();

Headless gotchas

If the above is enough for your scenario, then congrats! There are plenty of better things to do with your time than read the rest of this post.

For those with similar mental illness to me: headless Chrome unfortunately behaves differently from a normal browser window. When headless, it doesn’t use the GPU (compositing and rasterization happen in software), and frames aren’t tied to a real display’s vsync. This means performance numbers won’t technically match what a real user sees (real-world users with a GPU will for the most part see better numbers than you measure here, but like everything it depends on machine specs).

I suggest you just accept this as a fact of life, because the numbers you’re gonna get out of here are close enough for your purposes. This is SaaS, not something seriously life-altering like a game where you need super-accurate FPS. What you’re looking for here is tracking relative changes. If your headless FPS drops from 55 to 30 after a code change, something regressed. Don’t rely on headless for absolute values; check those yourself (on a variety of platforms and machine specs), and make sure you’re happy with how it feels live.

Further reading

These docs helped me, maybe they’ll help you too?