You can't. ActionFilters are for MVC only and don't run when using SignalR. The only common ground (as you realize) is ASP.NET itself. That means you can write an HttpModule but not an action filter.
You can learn more about SignalR security here: http://www.asp.net/signalr/overview/security