OK. I got it working a bit better by using KnowledgeBuilder to create a package(ResourceType.PKG) from the rules and using that as an URL resource for KnowledgeAgent. Yet, there were so many problems with this whole setup, so I ended writing my own implementation.
Basically what I have is a class that
- Looks for files in the external path
- Adds all rule files as resources to KnowledgeBuilder
- Adds knowledge packages from builder to KnowledgeBase
- Creates new stateless session from KnowledgeBase
The only thing I have in spring config is kBase, which I get with Spring nicely to my custom class
<drools:kbase id="ruleKnowledgeBase">
</drools:kbase>
Marking my own question answered as to me the case is closed.