bdd_widget_test

A BDD-style widget testing library

Why?

Isn't it cool to develop mobile apps in natural language? A language each of your team members can read and understand so that it involves everyone working on the project productively. While Dart is on its way to this goal, for tests there is a language for that! It's called Gherkin.

The aim of this library is in combining two effective and easy-to-use techniques: BDD(Gherkin) and widget testing.

Getting Started

Add the dependency

Add build_runner and bdd_widget_test dependencies to dev_dependencies section of the pubspec.yaml file.

dev_dependencies:
  build_runner:
  bdd_widget_test: <put the latest version here>
  ...

You may get the actual version from installation instructions on Pub site.

Write features

Create *.feature file inside test folder. Let's say you're testing the default Flutter counter app, the content might be:

Feature: Counter
    Scenario: Initial counter value is 0
        Given the app is running
        Then I see {'0'} text

Now ask build_runner to generate Dart files for you. You may do this with the command:

dart run build_runner watch --delete-conflicting-outputs

After that, the corresponding dart file will be generated for each of your feature files. Do not change the code inside these dart files as they will be recreated each time you change something in feature files.

During feature-to-dart generation additional step folder will be created. It will contain all steps required to run the scenario. These files will not be updated hence feel free to adapt the content according to your needs.

Run tests

You're good to go! bdd_widget_test generated plain old Dart tests, so feel free to run you tests within your IDE or using the following command

flutter test

Feature file syntax

Feature file sample:

# comment here

Feature: Counter

  Background:
    Given the answer is {42}

  After:
    Then clean up after the test

  Scenario: Initial counter value is 0
    Given the app is running
    Then I see {'0'} text

  Scenario: Plus button increases the counter
    Given the app is running
    When I tap {Icons.add} icon
    Then I see {'1'} text

Background and After sections are optional. A Background allows you to add some context to the scenarios that follow it. It can contain one or more Given steps, which are run before each scenario. An After scenarion run even if a test fails, to ensure that it has a chance to clean up after itself. Most probably you don't need to use this keyword.

Each feature file must have one or more Feature:s. Features become test groups in Flutter tests.

Each feature group must have one or more Scenario:s (or Example:s). Scenario become widget tests.

Each scenario must have one or more lines with steps. Each of them must start with Given, When, Then, And, But, or * keywords. Conventionally Given steps are used for test arrangements, When — for interaction, Then — for asserts. Keywords are not taken into account when looking for a step definition. You can have as many steps as you like, but it's recommended you keep the number at 3-5 per scenario. Having too many steps will cause it to lose it’s expressive power as specification and documentation.

Feature files are parsed with cucumber_gherkin, the official Dart Gherkin parser, so a file that is not valid Gherkin fails the build pointing at the offending file, line and column rather than being silently ignored. Stray text between scenarios, a second Background: in the same feature, and a keyword mistyped anywhere below a block's first step are all reported instead of quietly producing a test file that tests less than it appears to.

A description is still a description, though. Gherkin treats the lines between a keyword and the first step under it as description text, where any non-keyword line is legal, so a bullet list or a sentence opening with And under a Feature: or Rule: is kept as documentation as long as that block has steps under it. The cost is that a keyword mistyped in that same position — a scenario's first step, or a Scenrio: written directly under Feature: — is read as description too, and the steps under it are dropped without an error, unless the block ends up with no steps at all, in which case it is reported. Proofread the top of a block; below the first step, a typo is always caught.

The Scenario Outline keyword can be used to run the same Scenario multiple times, with different combinations of values.

A Scenario Outline must contain an Examples (or Scenarios) section. Its steps are interpreted as a template which is never directly run. Instead, the Scenario Outline is run once for each row in the Examples section beneath it (not counting the first header row).

The steps can use <> delimited parameters that reference headers in the examples table. The plugin will replace these parameters with values from the table before it tries to match the step against a step definition.

Scenario Outline example:

Feature: Sample

  Scenario Outline: Plus button increases the counter
    Given the app is running
    When I tap {Icons.add} icon <times> times
    Then I see <result> text

    Examples:
    | times | result |
    |    0  |   '0'  |
    |    1  |   '1'  |
    |   42  |  '42'  |

The Rule keyword groups related scenarios inside a feature. Each rule becomes a nested test group, and a rule may have a Background of its own that applies only to the scenarios it contains — it runs after the feature's Background, if there is one. Tags on a rule are inherited by its scenarios.

Feature: Counter

  Background:
    Given the app is running

  Rule: The counter never goes below zero

    Background:
      Given the counter is {0}

    Scenario: Minus button is disabled at zero
      Then I see disabled {'-'} elevated button

Here Minus button is disabled at zero runs the feature background first, then the rule background, then its own steps.

If you need to have the same step but with different parameters, you may use a DataTable-like syntax:

Feature: Sample

  Scenario: An answer
    Given the app is running
    When I enter <input> text into <field> text field
    | input      | field |
    | '42'       |   0   |
    | 'question' |   1   |
    Then I see {'Do not forget your towel!'} text

The above is equivalent to:

Feature: Sample

  Scenario: An answer
    Given the app is running
    When I enter {'42'} text into {0} text field
    And I enter {'question'} text into {1} text field
    Then I see {'Do not forget your towel!'} text

While the DataTable-like syntax is a good practice for scenarios that require repeated steps, for example, entering text in different fields, sometimes we want to prepare test data in a readable way and mock our scneario's prerequisites and assert the expected result in an explicit domain driven way. To handle this, we create a data table:

Feature: Search songs

  Scenario: Searched text matches a song's details
    Given available songs
    | 'artist'      | 'name'                      |
    | 'The Doors'   | 'Riders on the storm'       |
    | 'Bob Dylan'   | "Knockin' On Heaven's Door" |
    | 'The Beatles' | 'Here Comes the Sun'        |
    When I search for text {'door'}
    Then I see songs
    | 'artist'      | 'name'                      |
    | 'The Doors'   | 'Riders on the storm'       |
    | 'Bob Dylan'   | "Knockin' On Heaven's Door" |

For each of the above step lines that are followed by a table, in the related generate step file, the created function will have an object parameter of type DataTable:

import 'package:bdd_widget_test/data_table.dart' as bdd;
import 'package:flutter_test/flutter_test.dart';

/// Usage: Given available songs
Future<void> availableSongs(WidgetTester tester, bdd.DataTable dataTable) async {
  throw UnimplementedError();
}

Use the DataTable parameter to get access to the data:

final dataAsList = dataTable.asLists(); // [['artist', 'name'], ['The Doors', 'Riders on the storm'], ...]
final dataAsMaps = dataTable.asMaps(); // [{'artist: 'The Doors', 'name: 'Riders on the storm'}, ...]

Other languages

Gherkin keywords are available in 80 languages. Declare the language above the first feature keyword with a # language: header — below Dart header lines is fine — and write the keywords in that language:

# language: fr
Fonctionnalité: Compteur

  Contexte:
    Soit the app is running

  Scénario: La valeur initiale est 0
    Alors I see {'0'} text

Written below the first feature keyword the header is an ordinary comment, and the file is read in English. Step text itself is still whatever you write — only the keywords are translated. The one thing that does not travel is the step's name: its Dart file and function are built from the ASCII letters and digits in the step text, so Alors I see {'0'} text still becomes iSeeText in i_see_text.dart. Accents are folded onto ASCII first (los diacríticos son útil → los_diacriticos_son_util), and other scripts are fine alongside some ASCII text (Given アプリが起動している app is running → app_is_running). A step whose text holds no ASCII letter or digit at all — Given アプリが起動している on its own — names nothing Dart can compile, and fails the build naming the file, line and column, rather than writing out a step file called .dart. Rename the step, or give it a word of ASCII in it. The same goes for text that opens with a digit (Given 2FA is on → 2faisOn, which Dart refuses as a name), for text that opens with an underscore (Given _debug mode is on → _debugModeIsOn, a name Dart keeps private to the step file it is declared in, where the generated test cannot call it), and for a step named entirely by a parameter or an <outline placeholder>, since what is left once the parameters are out is nothing. The package's own extensions keep their spelling: After: is written in English in a French feature file as much as in an English one, and is rewritten internally into the keyword the file titles its scenarios with. See the full list of dialects for the keywords of each language.

Tags

Tags are used to filter scenarios in the test runner. Here are some examples:

@slow
@integration
Feature: Sample

  @important
  Scenario: An answer
    Given the app is running

Here we mark the test as slow, integration, and important.

Tags may share a line, as Gherkin allows: @slow @integration marks a feature as both. The one exception is this package's own @name: value tags — @testMethodName:, @testerType:, @testerName: and @scenarioParams: — whose value runs to the end of the line, so write one of those on a line of its own. Tags written before it on the same line are read as tags.

To run tests that are marked with @important tag, you can use the following command:

flutter test --tags important

To exclude tests that are marked with @slow tag, you can use the following command:

flutter test --exclude-tags slow

Gherkin standard

Feature files are parsed by cucumber_gherkin, the official Dart Gherkin parser, so anything valid in Gherkin is valid here — including Rule: and the keywords of all 80 languages. Doc strings (""") are parsed but currently ignored.

On top of the standard, this package adds a few extensions that make Flutter testing easier. They are not part of Gherkin, so other Cucumber tooling will not read them the way this package does:

Extension	What other Gherkin tools do with it
After: sections	Read as feature description text — the steps are silently dropped
A table under a step, used to repeat that step	Read as a DataTable argument, so the step runs once instead of once per row
Dart lines above Feature:	Parse error
Tags containing a space, e.g. @testMethodName: patrolTest	Parse error — a tag may not contain whitespace
Several Feature:s in one file	Parse error — Gherkin allows one feature per file
{} parameters, e.g. {'0'}	Nothing — to a Gherkin parser these are ordinary step text

The first two are worth singling out: they produce no error anywhere, the file just means something different.

Keeping your feature files canonical

If your feature files need to stay readable by other Cucumber tooling — a specification shared with a backend suite, a Gherkin editor plugin, a reporting tool — every extension except {} has a standard alternative this package already supports:

Instead of	Use
After:	Hooks — Hooks.afterEach runs after every scenario, failures included
Dart lines above Feature:	The customHeaders option in build.yaml
@testMethodName: patrolTest	@testMethodName:patrolTest — same meaning, no space, a valid tag
Several Feature:s in one file	One feature per file
A table under a step	Write the repeated steps out in full, as shown above

{} parameters need no replacement. A Gherkin parser treats them as part of the step text, so a feature file that uses them is still a valid Gherkin document — only your step definitions are specific to this package.

Predefined steps

This library comes with a list of predefined steps. They will be auto-generated for you, but you may want to adjust their implementation according to your needs.

List of predefined steps:

• I dismiss the page
• I don't see {..} icon
• I don't see {..} rich text
• I don't see {..} text
• I don't see {..} widget
• I enter {..} into {..} input field
• I see disabled elevated button
• I see enabled elevated button
• I see exactly {..} {..} widgets
• I see {..} icon
• I see multiple {..} texts
• I see multiple {..} widgets
• I see {..} rich text
• I see {..} text
• I see {..} widget
• I tap {..} icon
• I tap {..} text
• I wait
• The app is running
• The screen matches golden {..}

Hooks

If you want to add hooks, you need to add the addHooks flag to the build.yaml. This will generate a file that allows you to handle a beforeAll, afterAll, beforeEach and afterEach call. These hooks will be generated per directory, just like the steps. Also like with the steps, you can define a directory in the build.yaml to define one set location for the hooks. These hooks will then be used everywhere.

targets:
  $default:
    builders:
      bdd_widget_test|featureBuilder:
        options:
          addHooks: true
          hookFolderName: bdd_hooks

The beforeAll and afterAll do not take any properties, but the beforeEach and afterEach both provide the name and the tags of the feature. On top of this, the afterEach provides whether or not the test was successful.

FAQ

How to pass a parameter?

You may use curly brackets to pass the parameter into a step. The syntax is following:

  When I see {42} number
  And I see {Icons.add} icon

Notice, that the value inside brackets is copied to the Dart test file without changes hence it must be a valid Dart code. In the example above first step will have an int value. In order to pass a valid Dart string use '42' or "42".

You may call methods in step parameters, but most probably it's not what you want.

How to add additional imports to test files?

Most of the time you shouldn't do that, as the BDD tests simulate user's behavior and it's just not possible for users to know the implementation details. Nevertheless, sometimes it might be in hand, i.e. when you have custom domain models or components. For example, if you need to check Cupertino icons in the test, you may have:

import 'package:flutter/cupertino.dart';

Feature: ...
  Then I see {CupertinoIcons.back} cupertino icon

How to adjust linter for auto-generated tests?

Use the same trick as above, just write linter rules you wish to ignore at the beginning of the feature file:

// ignore_for_file: avoid_as, prefer_is_not_empty

Feature: ...

Any video tutorials on this?

Sure, you may find a BDD in Flutter playlist on youtube with the basic showcase.

How to test the UI? (golden tests)

BDD is UI agnostic, the main focus is on the requirements. If you need to test colors and layouts, a golden check is just another assertion, so the library comes with a predefined Then step on top of Flutter's own matchesGoldenFile:

Feature: Counter

  Background:
    Given the app is running

  Scenario: Initial counter screen
    Then the screen matches golden {'initial_counter'}

The generated step compares the whole screen with goldens/initial_counter.png. The scenario stays an ordinary testWidgets test, so Background:, After: and hooks work as usual, and no extra dependency or testMethodName override is needed. Run flutter test --update-goldens once to create the golden files, then flutter test to compare against them. See the example folder for a working setup.

Things to keep in mind:

• The golden path is resolved relative to the generated *_test.dart file, not to the step file. In test/sample.feature, the scenario above compares with test/goldens/initial_counter.png.
• Golden names must be unique among the features of one folder.
• The predefined step takes a WidgetTester, like every predefined step, so it doesn't fit features with @testerType: PatrolIntegrationTester. For those, write your own the_screen_matches_golden.dart in the step folder. The body doesn't use the tester, so it is the same with a PatrolIntegrationTester $ parameter. A step file that already exists always wins over the predefined one.
• The step captures the whole app. To capture a single widget, wrap it in a RepaintBoundary with a Key and write your own step that targets the boundary with find.byKey. Finding the widget alone usually still captures the whole screen, because the image is taken from the nearest RepaintBoundary at or above it.
• On a mismatch Flutter writes diff images to a failures/ folder next to the test. You will probably want to add it to .gitignore.
• By default Flutter tests draw every glyph as a box and load no icon fonts, so goldens show boxes instead of text and icons. To get real text, load fonts in a flutter_test_config.dart next to your tests. The example's test/flutter_test_config.dart loads the Roboto and Material Icons fonts that ship with the Flutter SDK, and hides the debug banner.
• Real fonts render slightly differently on each operating system. Either pick one OS for your goldens, e.g. the one your CI runs on, or keep one set per OS. For the latter, add a comparator that puts the OS in the path to your flutter_test_config.dart, and install it in testExecutable before await testMain(); with goldenFileComparator = _PerOsComparator(goldenFileComparator);. With it, goldens/initial_counter.png is stored as goldens/macos/initial_counter.png, goldens/linux/initial_counter.png and so on. See the example's test/flutter_test_config.dart, which does this next to loading fonts.

  import 'dart:io';
  import 'dart:typed_data';
  
  import 'package:flutter_test/flutter_test.dart';
  
  class _PerOsComparator extends GoldenFileComparator {
    _PerOsComparator(this._delegate);
  
    final GoldenFileComparator _delegate;
  
    // Rewrite the path only here: matchesGoldenFile passes the result to
    // compare and update, and shows it in failure messages.
    @override
    Uri getTestUri(Uri key, int? version) {
      final golden = _delegate.getTestUri(key, version);
      return golden.replace(pathSegments: [
        ...golden.pathSegments.take(golden.pathSegments.length - 1),
        Platform.operatingSystem,
        golden.pathSegments.last,
      ]);
    }
  
    @override
    Future<bool> compare(Uint8List imageBytes, Uri golden) =>
        _delegate.compare(imageBytes, golden);
  
    @override
    Future<void> update(Uri golden, Uint8List imageBytes) =>
        _delegate.update(golden, imageBytes);
  }

  flutter test sets up a comparator for each test file before it calls testExecutable, so wrapping it keeps the paths relative to the test file. Run flutter test --update-goldens on every OS to create its set; an OS without one fails with "Could not be compared against non-existent file". dart:io isn't available on the web, so this doesn't work with flutter test --platform chrome. The example works this way and ships macOS and Linux goldens.

How to reuse steps between projects?

You may reference any step using build.yaml file (see the example folder):

targets:
  $default:
    builders:
      bdd_widget_test|featureBuilder:
        options:
          externalSteps:
            - package:<your_package>/<your_step>.dart

If you have many packages you might want to reuse the whole list of external steps. For that you'll have to create a bdd_options.yaml file in the root folder of your project with the following content:

include: package:<package_a>/bdd_options.yaml # will include all steps defined in bdd_options.yaml of package_a
externalSteps:
  - package:<package_b>/<your_step>.dart
  - package:<package_c>/<your_step>.dart

Alternatively, if you need just to include an external config, use the include option in the build.yaml config:

targets:
  $default:
    builders:
      bdd_widget_test|featureBuilder:
        options:
          include: package:<your_package>/bdd_options.yaml

How to add custom headers to generated files?

You can add custom header lines (imports, comments, etc.) to all generated step files and feature files using the customHeaders option in the build.yaml file:

targets:
  $default:
    builders:
      bdd_widget_test|featureBuilder:
        options:
          customHeaders:
            - "import 'package:flutter_test/flutter_test.dart';"
            - "import 'package:patrol/patrol.dart';"
            - "// Custom test utilities"
            - "import 'package:my_custom_package/my_helper.dart';"

This is useful when you need to:

• Use custom test frameworks (like Patrol) instead of the default flutter_test
• Import custom helper classes or utilities in all your generated files
• Add specific packages that your tests will commonly use
• Include custom test utilities, mocks, or constants
• Add custom comments or documentation to generated files

The custom headers will be added to:

• Feature files: After any data table imports, replacing the default material_ui and flutter_test imports
• Step files: After any data table imports, replacing the default flutter_test import

How to group steps in a single project?

You may create sub-folders (like common, login, home, etc.) in the step folder and move generated steps there. The plugin is smart enough to find them (see the example folder).

I don't like the step folder name, how can I change it?

By setting the stepFolderName parameter with any name you like in the build.yaml file (see the example folder):

targets:
  $default:
    builders:
      bdd_widget_test|featureBuilder:
        options:
          stepFolderName: bdd_steps

That will tell the plugin to create a bdd_steps folder under test (instead of step near each feature) and put all the steps there.

You may set a relative path here (like ../../bdd_steps), just be sure that the target folder is still somewhere under the test folder.

How to write integration tests?

1. Add integration_test dependency to the pubspec.yaml file:

dev_dependencies:
  integration_test:
    sdk: flutter

2. Modify build.yaml file to support code generation in the integration_test folder like that (here is the full sample):

targets:
  $default:
    sources:
      - integration_test/**   # By default, build runner will not generate code in the integration folder
      - test/**               # so we override paths for code generation here
      - lib/**
      - $package$

3. (Optional) If you plan to re-use steps between integration and widget tests set a common step folder in the build.yaml file like that (here is the full sample):

stepFolderName: step

4. Done. Now you may create feature files in integration_test folder. You may want to review the official documentation for instructions on how to run integration tests.

Patrol tests

Patrol is a popular testing framework for Flutter. It provides a possibility to control native UI from the Dart code, and also provides a new custom finder that simplifies the development process of the widget tests. You may use Patrol together with bdd_widget_test to get the best from both worlds.

Here is an example of a BDD scenario with Patrol framework:

import 'package:patrol/patrol.dart';

@testMethodName: patrolTest
@testerName: $
@testerType: PatrolIntegrationTester
Feature: Counter
    
    Background:
        Given the app is running
    
    Scenario: Initial counter value is 0
        Then I see {'0'} text

    @scenarioParams: nativeAutomation: true
    Scenario: Add button increments the counter
        When I tap {Icons.add} icon
        Then I see {'1'} text

Notice how the second scenario is annotated with nativeAutomation enabling you to control native UI elements. Setting the tester type to PatrolIntegrationTester allows you to use the power of the custom finder introduced by Patrol.

To streamline your testing setup, you can conveniently configure key parameters in the build.yaml file if you plan to use Patrol in all your tests:

targets:
  $default:
    builders:
      bdd_widget_test|featureBuilder:
        options:
          testMethodName: patrolTest
          testerName: $
          testerType: PatrolIntegrationTester
          includeIntegrationTestBinding: false
          customHeaders:
            - "import 'package:flutter_test/flutter_test.dart';"
            - "import 'package:patrol/patrol.dart';"

Since Patrol version 3.0.0, IntegrationTestWidgetsFlutterBinding.ensureInitialized must not be called. Set includeIntegrationTestBinding to false.

I don't like that the plugin creates steps under test folder. How to change that?

You may set a relative path in the build.yaml file (see the example folder):

relativeToTestFolder: false
stepFolderName: integration_test/steps # if you want to have steps in the integration_test folder
hookFolderName: integration_test/bdd_hooks # if you want to have hooks in the integration_test folder

Can AI coding agents learn this dialect?

Yes. The package ships an agent skill, bdd-widget-test-gherkin, in its skills/ folder. It covers feature file syntax, step reuse, tables, tags, step files, hooks, configuration, and golden and Patrol setups. Install it into your project with:

dart run skills@ get -p bdd_widget_test

Contributing

If you find a bug or would like to request a new feature, just open an issue. Your contributions are always welcome!

License

bdd_widget_test is released under a MIT License. See LICENSE for details.