Timed Testing - @let@token TORX --- A (t)ioco-based Testing Tool

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Timed Testing
TORX — A (t)ioco-based Testing Tool
Lehrstuhl Informatik 2 (MOVES)
RWTH Aachen
Summer Semester 2007
1 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
2 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification-Based Testing
System
specification
Formal
model
Test-Case
Generator
Test-
Executor
pass/fail
Test-Suite
3 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification-Based Testing
System
specification
Formal
model
Test-Case
Generator
Test-
Executor
pass/fail
Test-Suite
Implementation
according to
Model
3 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification-Based Testing
We have covered:
Formal models up to timed models
relations between formal specifications and formal
implementations
test cases, test suits
test execution
We never left the formal world
4 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
TorX: A testing tool based on the (t)ioco(M) theory
5 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
6 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Architecture
Specification
Explorer Primer Driver Adaptor IUT
7 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Explorer
Specification
Explorer Primer Driver Adapter IUT
Responsible to translate specification in LTS
representation
exchangeable component
Explorers for: LOTOS, Promela, mCRL, timed automata
8 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Primer
Specification
Explorer Primer Driver Adapter IUT
Implements test-case generation
Specifications given as LTS (explorer)
On-the-fly testing: Test case-generation and -execution are
done simultaneously
A new part of the test case is generated by need
casts verdicts
9 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Driver
Specification Explorer Primer Driver Adapter IUT
Control component (motor) of the whole tool chain:
GUI
Decides how to proceed with testing (input/output),
conulting the adapter
negotioates between primer and adapter
10 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Adapter
Specification
Explorer Primer Driver Adapter IUT
The interface between the formal world and the physical
world
translates abstract actions into concrete actions:
a i? ∈ Act I might be translated to an RPC to the IUT
The output of the rpc might be translated back to an output
o! ∈ Act U
records occurrence times of inputs and outputs
generates quiescence signals after timeout M
Examples: http server, HTML browser engine, GUI,
network component
11 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
IUT
Specification
Explorer Primer Driver Adaptor
IUT ˆ= Implementation Under Test
is supposed to behave like the Specification allows
must obey the testing hypothesis: can be modeled as
ioco : IOTS
tiocoM: an input-enabled, M-quiescent TLTS, where M is
known
Examples: http server, HTML browser engine, GUI,
network component
IUT
12 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
13 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification formalism: timed automata
Specification: a coffee machine
Implementation: a timed automaton simulator
Testing tool: TorX
14 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Timed Automata
What are Timed Automata?
Think: LTS with some extras:
Clocks
Guards
Invariants
Resets
=⇒ some slides from JP Katoen (Advanced Model Checking)
15 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Timed Automata
What are Timed Automata?
Think: LTS with some extras:
clock x
l
x ≤ 2
i?/o!/τ/δ
x ≥ 1
x := 0
l ′
15 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Timed Automata
What are Timed Automata?
Think: LTS with some extras:
0
1
i?/o!/τ
2
15 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification: a Coffee Machine
16 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification: a Coffee Machine
Delta!
M

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Specification: a Coffee Machine
Skip to UPPAAL
16 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Implementation: simulating a timed automaton
Skip to simulator
17 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Testing
Skip to demo
18 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Testing: A Mutant
Delta!
M

Introduction TorX A demo in timed testing Preparing for the/an oral exam
What to do with this?
Confession:
This course was advertisement for the line of research I am
doing.
(fishing for diploma/master students)
Possible Master/Diploma projects:
General Theme: timed testing
Practical: Case studies
theoretical work
with or without accompanying tool development
20 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
21 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
22 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Overview over the course:
1 Specifiying Labelled Transition Systems
2 Preorders, the linear-time/branching time spectrum
3 Observing Behaviour: preorders, observers, etc.
4 Towards “real” testing: Conformace testin with inputs and
outputs: ioco
5 Timed Testing: tiocoM
23 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
24 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Disclaimer:
THIS LIST IS NOT
EXHAUSTIVE!!!
25 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
In general
Identify similarities
Quite some similarities between the presented topics:
blatantly obvious in Chapter 3 (≤tr , ≤ f , ≤ ft, . . . )
Structure of, e.g., induction proofs
. . .
Identify the really important bits
Might Theorem 4.6.1 be of importance? The only theorem
with a complete proof in the whole script? (Read this as a
rhetorical question).
. . . and know them
26 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
About the exercises
Do not just memorise the solutions
try to understand and replicate them
you should be able to tackle (at least in principle) similar
problems in the exam (proof ideas and such).
27 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1) Specifiying Labelled Transition Systems
Basic definitions
Processes: Syntax/Semantics
non-determinism
28 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
2) Preorders etc
Basic definitions (preorders, corser/finier etc.)
Trace preorder
Bisimulation
linear time-branching time spectrum: what does it depict?
relation to non-determinism
29 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
3) Observing Behaviour
Basic definitions of the preorders
Where to find them in the lt-bt-spectrum?
Observers, Observations, relations to preorders,
proof(-idea)s etc.
relevant examples
The problem with τ-steps and divergence
30 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
4) Conformance testing with inputs and outputs
Basic definitiosn
Concepts like input-enbledness, quiescence, and where
they are needed.
The four definitions of iot
ioconf, ior, ioco, differences, etc.
why are we not content with ior, why ioco
31 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
5) Timed Testing
This Chapter is
Yet . . .
the most complicated
the least developed (no proofs)
Basic definitions (TLTS)
meaning of time-divergence, input-patience etc.
definition tiocoM
idea behind test cases and the test case-generation
algorithm
32 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
1 Introduction
2 TorX
3 A demo in timed testing
4 Preparing for the/an oral exam
Overview
What do I expect from you to know?
How to prepare
33 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
Important to remember
Important to remember
You have to tell what you know (with the help of pencil and
paper)
You must not only know the answers, but know how to tell
the answers
How to prepare
practice to tell somebody what you know
Learn with somebody else
Generate questions
Ask them and let others try to answer them
34 / 35

Introduction TorX A demo in timed testing Preparing for the/an oral exam
In case of questions:
Availability
Until September
From October on
35 / 35