HTTP response code when requested websocket subprotocol isn't supported/recognized

Question

When upgrading an HTTP connection to a websocket, one can provide one or more subprotocols in the optional HTTP header 'Sec-WebSocket-Protocol'.

If the server accepts any of the subprotocols it responds with HTTP response code 101 ("HTTP/1.1 101 Switching Protocols") and includes the HTTP header 'Sec-WebSocket-Protocol' indicating the selected subprotocol.

But how should the server correctly handle an unknown/unsupported subprotocol?

Should this be done 'within' the HTTP connection -- by use of some HTTP response code?

Or should the connection be upgraded to a websocket -- and immediately be closed by the server by sending a 'Close Frame' with some of the predefined websocket Status codes?

What does the RFC6455 say? I cannot come to a conclusion. How does existing server implementations handle it?

Regards /Per/

Answer 1

From a brief glimpse at RFC 6455, I believe the WebSocket Subprotocol is optional and negotiated. In section 4.2.2, under "Server Rquirements":

   /subprotocol/
      Either a single value representing the subprotocol the server
      is ready to use or null.  The value chosen MUST be derived
      from the client's handshake, specifically by selecting one of
      the values from the |Sec-WebSocket-Protocol| field that the
      server is willing to use for this connection (if any).  If the
      client's handshake did not contain such a header field or if
      the server does not agree to any of the client's requested
      subprotocols, the only acceptable value is null.  The absence
      of such a field is equivalent to the null value (meaning that
      if the server does not wish to agree to one of the suggested
      subprotocols, it MUST NOT send back a |Sec-WebSocket-Protocol|
      header field in its response).  The empty string is not the
      same as the null value for these purposes and is not a legal
      value for this field.  The ABNF for the value of this header
      field is (token), where the definitions of constructs and
      rules are as given in [RFC2616].

The server should not send a subprotocol response header with a value other than 'null' if it did not agree to use the subprotocol with the client, and it is the client's responsibility to either continue or terminate the connection at that point.

Answer 2

The whole point of WebSocket is to use WebSocket to run a specific protocol from the client to the server. But the client and server need to know the protocol so that both ends know what to do with the WS messages coming through. The WS message handling is no different than what you would do at a TCP level. You don't have to use the protocol field, it's only relevant if you want to enforce compatibility.

For the Kaazing server, we provide specific protocol libraries that will sit on top of the basic websocket layer. You can also find various protocol libraries on Github. With the exception of the handshake, everything else is the way you would handle it with TCP. You process the websocket messages and try to decode the protocol. The protocol field in the handshake should be used to make sure that your library is actually the matching protocol library. So for example if my server speaks STOMP, and I try to connect with my client and I want to speak STOMP, my STOMP library should check the protocol field to make sure it matches. The endpoints can indicate in the order of preference for example that it can speak AMQP 0.9 and AMQP 1.0, and will choose AMQP 1.0 if both are available. If one of the endpoints doesn't speak any of the protocols the other end can speak, it can return null and thus the connection is terminated.