I found it. I had to put an h:messages tag somewhere and update it using a button. My form in XHTML now:
<h:form>
<h:messages id="errorMessages" style="color:red;margin:8px;" />
<br></br>
<p:panelGrid columns="2" style="margin-bottom:10px" cellpadding="5">
<h:outputText value="Item: "/>
<p:selectOneMenu value="#{devTestController.items}">
<f:selectItems value="#{devTestController.items}" var="item" itemLabel="#{item.label}" itemValue="#{item.value}"/>
</p:selectOneMenu>
</p:panelGrid>
<p:commandButton value="asdf1" update = "errorMessages" action="#{devTestController.doAction()}"/>
</h:form>
Now I get the messages when I press the button:
It seems that I am missing a converter.