To make test case definitions more readable across different layers of the company (testers, developers, project management, ...), and to simplify the step from defining test cases to implementation of test cases, the Gherkin language was chosen to create our test specs in a BDD (Behavior Driven Development) framework.
This enables us to write specifications in a more natural language where test steps are linked to executable code and it's parameters. This is done with matchers that try to find a step definition matching a template (ex. a regular expression) which is linked to an executable function. Arguments can be passed to these functions by defining in the template which parts of the steps are to be used as input for the variable.
In a more complex environment like a train, many inputs and outputs are available, including many with the same functionality (an announcement can be made from multiple places in the train). This poses some difficulties to map the variables and their expected values detected by the Gherkin parser on the resources represented by them. As step definitions are preferably relatively short, it is undesirable to enter big lists here containing all options. To be able to do this, those lists are often passed in a more descriptive way.
Some examples:
When pressing the PA pushbutton in any other cab
Then the CC jingle should be heard on saloon loudspeakers in all zones
Then the CC jingle should be heard on cab loudspeakers in all zones except the calling cab
Then the announcement should be heard in all active PA zones
Then the HMI interface should show the correct calling pattern starting with the next station
Then PIS should report the active headcode to the TCMS
(PA = Public Announcement, CC = Crew Call, HMI = Human Machine Interface, TCMS = Train Control & Management System, PIS = Passenger Information System)
If the step matchers in Gherkin are well written, Gherkin is good at connecting the steps written as natural language to functions, and to pass the sub-strings describing the variable as arguments to those functions. From that point onward, it's up to whoever implements the function to make sure those strings can be understood by the function. As these strings are still (descriptive) strings, they still need to be connected to the correct input/output/state of the system somehow.
For simple cases, a simple mapping table can resolve this, but cases where we need to select subsets based on state (ex. all active zones), or based on previous actions or results (all/any other cabs), are not so easy to map to the correct resources.
This poses two problems:
What are efficient ways to define all inputs/outputs/states in a big test environment representing a train, in such a way, that we can easily that we can easily find a subset matching a certain criteria?
Structure?
Storage format?
Usability by non-experts?
Retrieval of data?
What are efficient ways to translate descriptive strings that refer to subsets of components of our system to queries that can extract the needed input/outputs/states from such a definition with as little effort as possible for who write test cases?
Static mapping?
Writing code matching the purpose?
Natural language tools/libraries?
Hybrid options with the above?
The environment in which this needs to work is Python 3.7+, as this is what is used in our current test framework. The goal is to define models for test setups with test automation capabilities, and to provide a library in the framework to map the natural language to these models.
Nature of the work
Level: Master
Specialty: AI / Machine Learning, Automation, Software
Type of work: Research: 30%, Implem.: 30%, Experim.: 40%
Location: Televic, University
Type of activities: Experimenting, Implementation, Literature Study
Apply now