learning_results.tex 9.43 KB
 Paul Fiterau Brostean committed Feb 06, 2017 1 2 3 \section{Learning results} \label{sec:result} \newcommand{\dk}{\emph{k}} We use the setup described in Section~\ref{sec:setup} to learn models for OpenSSH, BitVise and DropBear SSH server implementations.  Paul Fiterau Brostean committed Feb 08, 2017 4 OpenSSH represents the focal point, as it is the most popular implementation of SSH (with over 80 percent of market share in 2008~\cite{Albrecht2009Plaintext})  Paul Fiterau Brostean committed Feb 06, 2017 5 6 7 and also, the default server for many UNIX-based systems. DropBear is an alternative to OpenSSH designed for low resource systems. BitVise is a well known proprietary Windows-only SSH implementation.  Paul Fiterau Brostean committed Feb 14, 2017 8 In our experimental setup, the {\dlearner} and {\dmapper} were running inside a Linux Virtual Machine. OpenSSH and DropBear were  Paul Fiterau Brostean committed Feb 06, 2017 9 learned over a localhost connection, whereas BitVise was learned over a virtual connection with the Windows host machine.  Paul Fiterau Brostean committed Feb 06, 2017 10 11 Certain arrangements had to be made including the setting of timing parameters to fit each implementation.  12 13 \begin{figure*} \centering  Paul Fiterau Brostean committed Feb 14, 2017 14  \includegraphics[scale=0.30]{ssh-server}  15 16 17 18 19  \caption{OpenSSH server. States are distributed into 3 clusters, one for each layer, plus a state for when connection was lost. We eliminate redundant states and information induced by the {\dmapper}, as well as states present in successful rekey sequences. Wherever rekey was permitted, we replaced the rekey states and transitions by a single \textsl{REKEY SEQUENCE} transition. We also factor out edges common to states within a cluster. We replace common disconnecting edges, by one edge from the cluster to the disconnect state. Common self loop edges are colored, and the actual i/o information only appears on one edge. Transitions with similar start and end states are joined together on the same edge. Transition labels are kept short by regular expressions(UA\_* stands for all inputs starting with UA\_) or by factoring out common start strings. Green edges highlight the happy flow. } \label{fig:sshserver} \end{figure*}  Paul Fiterau Brostean committed Feb 14, 2017 20 21 OpenSSH was learned using a full alphabet, whereas DropBear and BitVise were learned using a restricted alphabet. Both versions of the alphabets are described in Subsection~\ref{subsec:alphabet}. The primary reason for using a restricted alphabet was to reduce learning times.  Paul Fiterau Brostean committed Feb 08, 2017 22 23 Most inputs excluded were inputs that either didn't change behavior (like \textsc{debug} or \textsc{unimpl}), or that proved costly time-wise, and were not critical to penetrating all layers. A concrete example is the user/password based authentication inputs (\textsc{ua\_pw\_ok} and  24 25 \textsc{ua\_pw\_nok}). It would take the system 2-3 seconds to respond to an invalid password, perhaps in an attempt by the designers to thwart brute force attacks. By contrast, public key authentication resulted in quick responses. The \textsc{disconnect} input presented similar  Paul Fiterau Brostean committed Feb 14, 2017 26 challenges, as it would take a varying amount of time until the system responded. This was particularly problematic for BitVise.  Paul Fiterau Brostean committed Feb 08, 2017 27 28 29 30 31 32 33  %Not only that, but failed password authentication %attempts are also likely to trigger security mechanisms that would block subsequent authentication attempts. While this is %As an example, \textsl{ua\_pw\_ok} contours the same behavior as \textsl{ua\_pk\_ok}. But while authenticating %with a public key was done quickly, authenticating with a username/password proved time consuming (it would take the system 2-3 seconds to respond to %invalid credentials \textsl{ua\_pw\_ok}). The \textsl{disconnect} proved expensive in a similar way.  Paul Fiterau Brostean committed Feb 06, 2017 34 35  For testing, we used random and exhaustive variants of testing algorithm described in  Paul Fiterau Brostean committed Feb 06, 2017 36 37 38 39 40 \cite{SMJV15}, which generate efficient test suites. Tests generated comprise an access sequence, a middle section of length {\dk} and a distinguishing sequence. The exhaustive variant for a set {\dk}, generates tests for all possible middle sections and all states. Passing all tests provides some notion of confidence, namely, that the learned model is correct unless the (unknown) model of the implementation has at least {\dk} more states. The random variant produces tests with randomly generated middle sections. No formal confidence is provided, but past experience shows this to be more effective at finding counterexamples since {\dk} can be set to higher values. We executed a random test suite with {\dk} of 4 comprising 40000 tests for OpenSSH, and 20000 tests for BitVise and DropBear.  Paul Fiterau Brostean committed Feb 08, 2017 41 We then ran an exhaustive test suite with {\dk} of 2 for for all implementations.  Paul Fiterau Brostean committed Feb 06, 2017 42   43   Paul Fiterau Brostean committed Feb 06, 2017 44 Table~\ref{tab:experiments} describes the exact versions of the systems analyzed together with statistics on learning and testing, namely:  Paul Fiterau Brostean committed Feb 10, 2017 45 (1) the number of states in the learned model, (2) the number of hypotheses built during the learning process and (3) the total number of learning and test queries run. For test queries, we only consider those run on the last hypothesis.  Paul Fiterau Brostean committed Feb 06, 2017 46   Paul Fiterau Brostean committed Feb 10, 2017 47 48 49 %BitVise: MemQ: 24996 TestQ: 58423 %Dropbear: MemQ: 3561 TestQ: 30629 %OpenSSH: MemQ: 19836 TestQ: 76418  Paul Fiterau Brostean committed Feb 06, 2017 50 51 52  \begin{table}[!ht] \centering  Paul Fiterau Brostean committed Feb 10, 2017 53 \small  Paul Fiterau Brostean committed Feb 06, 2017 54 55 \begin{tabular}{|l|l|l|l|l|l|l|} \hline  Paul Fiterau Brostean committed Feb 10, 2017 56 57 58 59 {\centering{\textbf{SUT}}} & \textbf{States} & \textbf{Hypotheses } & \textbf{Mem. Q.} & \textbf{Test Q.}\\ \hline %& \textbf{Tests to last Hyp.} & \textbf{Tests on last Hyp.} \\ \hline OpenSSH 6.9p1-2 & 31 & 4 & 19836 & 76418 \\ %& 1322 & 50243 \\ BitVise 7.23 & 65 & 15 & 24996 & 58423 \\ %& 9549 & 65040 \\ DropBear v2014.65 & 17 & 2 & 19863 & 76418 \\ \hline %& 15268 & 56174 \\  Paul Fiterau Brostean committed Feb 06, 2017 60 61 62 63 64 65 66 \end{tabular} \caption{Statistics for learning experiments} \label{tab:experiments} \end{table} The large number of states is down to several reasons. First of all, some systems exhibited buffering behavior. In particular, BitVise would queue  Paul Fiterau Brostean committed Feb 08, 2017 67 responses for higher layer inputs sent during key re-exchange, and would deliver them all at once, after the exchange was done. Re-exchanging keys (rekey-ing) was also  Paul Fiterau Brostean committed Feb 14, 2017 68 a major contributor to the number of states. In states allowing rekey, following the sequence of transitions comprising the rekey should lead back to the starting state. This  69 70 leads to 2 additional rekey states for every state permitting rekey. A considerable number of states were also added due to {\dmapper} generated outputs such as \textsc{ch\_none} or \textsc{ch\_max}, outputs which signal that no channel is open or that the maximum number of channels have been opened.  Paul Fiterau Brostean committed Feb 14, 2017 71 Figure~\ref{fig:sshserver} shows the model learned for OpenSSH, with various changes applied to improve readability. The happy flow, contoured in green, is fully explored in the model and mostly matches our earlier description of it\footnote{The only exception is in the transport layer, where unlike in our happy flow definition, the server is the first to send the \textsc{newkeys} message. This is also accepted behavior, as the protocol does not specify which side should send \textsc{newkeys} first.}. Also explored is what happens when a rekey sequence is attempted. We notice that rekey is only allowed in states of the Connection layer. Strangely, for these states, rekey is not state preserving, as the generated output on receiving a \textsc{sr\_auth}, \textsc{sr\_conn} or \textsc{kex30} changes from \textsc{unimpl} to \textsc{no\_resp}. This leads to two sub-clusters of states, one before the first rekey, the other afterward. In all other states, the first step of a rekey (\textsc{kexinit}) yields (\textsc{unimpl}), while the last step (\textsc{newkeys}) causes the system to disconnect.  72   Paul Fiterau Brostean committed Feb 14, 2017 73 74 We also found strange how systems reacted to \textsc{sr\_conn}, request for services of the Connection layer. These services can be accessed once the user had authenticated, without the need of an explicit service request. That in itself was not strange, because authentication messages already mention that connection services should start after authentication \footnote{This is a technical detail, the message format of authentication messages requires a field which says the service started after authentication. The only option is to start Connection layer services. }. What was strange was that, making this explicit request either lead to \textsc{unimpl}/\textsc{no\_resp} with no state change, as in the case of OpenSSH, or termination of the connection, as in the case of \textsc{BitVise}. The latter was particularly strange, as in theory, once authenticated, the user should always have access to the service, and not be disconnected on requesting this service. Only \textsc{DropBear} seems to respond positively (\textsc{sr\_accept}) to \textsc{sr\_conn} after authentication.  Paul Fiterau Brostean committed Feb 14, 2017 75 76  We also notice the intricate authentication behavior, it seems that password authentication is the only form that allows you to authenticate after an unsuccessful attempt. Finally, of all implementations tested, only BitVise allowed multiple terminals to be requested over the same channel. As depicted in the model, OpenSSH abruptly terminates on requesting a second terminal. DropBear exhibits a similar behavior.  77 78 79   Paul Fiterau Brostean committed Feb 08, 2017 80 81   Paul Fiterau Brostean committed Feb 07, 2017 82 83 84 85 %To give a concrete example, the {\dmapper} on every \textsl{ch\_open} saves a channel identifier and sends %a corresponding message to the {\dsut}. If \textsl{ch\_open} is called again, the {\dmapper} responds with a \textsl{ch\_max}. The channel identifier is removed %by a \textsl{ch\_close} input leading to pairs of identical states with and without the channel identifier, even in states where channels are not relevant (like for example states prior to authentication).  Paul Fiterau Brostean committed Feb 06, 2017 86