Commit f1f63ffb authored by Joshua Moerman's avatar Joshua Moerman
Browse files

Adds a part on the rers challenge in the intro

parent b32af5e7
......@@ -89,9 +89,10 @@ In a test query, similarly to a membership query, the learner asks for the syste
If the system's response is the same as the predicted response (by the hypothesis) for all test queries, then the hypothesis is assumed to be equivalent to the target.
Otherwise, if there is a test for which the target and the hypothesis produce different outputs, then this input sequence can be used as a counterexample.
One of the main advantages of using conformance testing is that it can discover all counterexamples for a hypothesis with $n$ states, under the assumption that the system can be modelled by a target FSM with at most $m$ states, $m \leq n$.
One of the main advantages of using conformance testing is that it can efficiently identify the hypothesis from the set of all finite state machines of size at most $m$.
This means that if we know a bound $m$ for the size of the system we learn, we are guaranteed to find a counterexample.
Unfortunately, conformance testing has some notable drawbacks.
First, it is hard (or even impossible) in practice to determine an upper-bound on the number of states of the system's target FSM ($m$).
First, it is hard (or even impossible) in practice to determine an upper-bound on the number of states of the system's target FSM.
Second, it is known that testing becomes exponentially more expensive for higher values of $m$ \citep{Vasilevskii1973}. %, as a complete test set should include each sequence in the so-called \emph{traversal set}, which contains all input sequences of length $l = m - n + 1$ .
%Moreover, this set should be applied to each state of the hypothesis.
%Therefore, in practice, often a small value for $l$ is chosen, in the hope that the test set will at least contain one counterexample that can be used to increase the size of the hypothesis.
......@@ -118,8 +119,23 @@ The fittest test cases can then be used as a source for mutation-based fuzzing.
Hence, tests are mutated to see if the coverage of the program is increased.
Iterating this process creates an evolutionary approach which proves to be very effective for various applications \citep{afl-website}.
In this paper, we use such an evolutionary fuzzing approach to generate a test set for use in model learning.
This test set is combined with that of a traditional (albeit slightly modified) conformance testing method for implementing equivalence queries.
\subsection{RERS Challenge 2016}
In this paper we report on our experiments in which we apply above techniques to the RERS challenge 2016.
The challenge consists of two parts: 1) problems for which we have to prove or disprove certain properties and 2) problems for which we have to find the reachable error states.
The problems are provided as source code (either C or Java) which are derived from some FSM.
To describe our approach briefly, we used a state of the art learning algorithm with a decent conformance testing algorithm.
Furthermore we used a fuzzer to generate potentially interesting traces.
For part (1) of the challenge the fuzzer did not find any counter examples.
For part (2) it did, which resulted in more error states being reached.
Due to time constraints we isolated learning and fuzzing.
We plan to experiment with integrating fuzzing in the learning loop.
%\subsubsection*{Related work}
%\citet{Duchene2012} have used a combination of model learning and evolutionary mutation-based fuzzing to detect web injection vulnerabilities.
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment