I think I have this one figured out. I failed to set the heartbeat configuration on the STOMP connection. Once I set these values I began seeing heartbeats sent to the client by the server, and when I pulled the plug on the web socket application the heartbeats stopped, as they should. After that point is was very easy to implement some reconnect logic based on the last time I received a heartbeat and if it was too old or not. Here is some sample code for configuring the STOMP client, most of this I pulled from the spring stock-portfolio stomp client example.
Inside the StompWebSocketHandler class you simply add this block of code. You would obviously set the heartbeatInterval variable to whatever value you desire.
public void afterConnectionEstablished(WebSocketSession session) throws IOException {
StompHeaderAccessor headers = StompHeaderAccessor.create(StompCommand.CONNECT);
headers.setAcceptVersion("1.1,1.2");
headers.setHeartbeat(heartbeatInterval, heartbeatInterval);
Message<byte[]> message = MessageBuilder.withPayload(new byte[0]).setHeaders(headers).build();
TextMessage textMessage = new TextMessage(new String(encoder.encode(message), DEFAULT_CHARSET));
session.sendMessage(textMessage);
}