So as I wrote in the "edit" of the question, it turns out it's a conflict because this same JavaSE project also has dependencies to glassfish embedded so that it can make JMS requests to a glassfish server:
<dependency>
<groupId>org.glassfish.main.extras</groupId>
<artifactId>glassfish-embedded-all</artifactId>
<version>3.1.2.2</version>
</dependency>
I guess the weld libraries from glassfish-embedded and from weld-se are fighting with each other or confusing the debugger. Didn't manage to make this work for now.