CSS:regression



Quick Intro

  • Rob Pocklington
  • Java, Ruby, Groovy, Scala, Node + JS
  • Worked at MYOB, Sportsbet, Seek and NAB
  • Frontend Development Lead @ AGL

  • Breakdown

  • Overview
  • Tool + Features
  • Setup
  • Demo
  • How to Automate
  • Discoveries
  • Closing Thoughts

  • Overview

  • You've got a clean codebase                                  ✅
  • You've automated your unit tests                           ✅
  • You have a good code review practice                    ✅
  • You've automated the most important user flows    ✅


    • Before Deployment
    After Deployment

  • Overview (cont ...)

  • Prod isn't broken - but it doesn't look quite right, why?

  • It's not always the Javascript or logic that blows up
  • ... it's the HTML
  • ... or the CSS



  • Testing margin and padding
    isn't that important

  • ... unless it causes your login button to disappear
  • or legal terms and conditions are off-screen or ellip...



  • It's 2018 - it's a jungle out there!

  • Hardware, OS, screen-size, pixel density, browser engine ...
  • You can't test everything

  • ... but if your only focus is testing functionality
  • ... the visuals can still be broken

  • Backstop.js

  • Screenshotter / Comparison Tool / Reporter

  • Uses Resemble.js

  • Features

  • Takes Screenshosts: with Firefox or Chrome
  • Multi-platform: works on Windows, Mac or Linux
  • Custom breakpoints: (mobile, tablet, desktop etc.)
  • Filter (include / exclude) capture regions via CSS selectors


  • Features (cont ...)

  • Single Page App compatible
  • Supports cookies, custom event hooks, waits or ready checks
  • Very quick - runs browser instances in parallel
  • Customisable threshold levels for comparisons

  • JUnit Report - good for CI
  • HTML Report - better for humans =)

  • Setup

  • One config file  backstop.json
  • An array of scenarios to capture screenshots

  • Run backstop reference to capture reference screenshosts
  • Run backstop test to capture and compare screenshosts
  • Run backstop approve to accept changes


    • Demo





    How to Automate

    How to Automate (cont ...)

  • Fail the build if  backstop test  fails
  • Publish the HTML Report as an artifact on the build
  • Developer inspects the changes, approves or fixes issues

    • Discoveries

  • May need multiple backstop.json files for various reasons
  • Font rendering is significantly different between
    Win / Mac / Linux :(

  • Solution #1 - run all Backstop.js tasks in Docker
  • Solution #2 - have a CI task to approve and commit changes

    • Ideas and Future plans

  • Test a version of the site without CSS (HTML only)
  • Emulate IPhone and Android devices
  • Dynamically build backstop.json files
  • Auto-publish comparison reports to a Slack channel


    • Closing thoughts

  • Of all these tools, Backstop.js has the best reports
  • it has an active and well-maintained repository

  • Automates a time-consuming but important task in our process
  • Enables easier visual reviews for all team members


    • Questions?






    Thanks for listening!

  • Slides
    rp.js.org/css-regression-pres/

    Twitter
    @melbjs @rpocklin
  • AGL are hiring (Javascript / Typescript / .NET Developers and Testers)