flake

Identify problematic jest tests

This is a Bash CLI script for identifying "flaky" jest tests

Usage

./flake.sh [-s jest_script] [-i iterations] [-j] [-h]

note In a CI environment, it is convenient to run

flake_ci.sh

as this file has no dependencies

Options

• -s jest_script

  : Specify the jest test script (default: npm run test -- --json)

• -i iterations

  : Specify the number of test iterations (default: 100)

• -d directory

  : Specify the directory to run the test script (default: current directory)

• -j

  : Save jest --json outputs (default: false)

• h

  : Display a help message

Analyze

--json

output files

If this script hangs or fails, it can be useful to run an analysis on the persisted

--json

output files generated from this CLI

./flake_on_files.sh [-d directory] [-h]

Options

• -d directory

  : Specify the directory where the jest JSON files reside (default: current directory)

• h

  : Display a help message

Github Action

name: Flake

on:
  schedule:
    - cron: "0 0 * * 0"

jobs:
  run-flake:
    runs-on: ubuntu-latest
    timeout-minutes: 60

    steps:
      - name: Clone project
        uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: "18"

      - name: Install npm
        run: npm install -g npm

      - name: Install dependencies
        run: npm install

      - name: Checkout flake repo
        uses: actions/checkout@v3
        with:
          repository: neo-andrew-moss/flake
          ref: main
          path: flake

      - name: Run flake
        run: |
          chmod +x ./flake/flake_ci.sh
          ./flake/flake_ci.sh -s 'npm run test -- --silent --json --maxWorkers=4' -i 100 > flake_logs.txt

      - name: Print results
        run: |
          echo "Flake results:"
          cat flake_logs.txt

note This uses a lot of CI minutes. Be careful.