I think I see the problem. POI does support Conditional Formatting and it reads them correctly only when the value is not a reference to another cell. So for example if I use the rule Format only cells that contain and select Specific Text and containing and a reference to a cell (=$A$1) then it won't be read by POI. If I change the reference to a static value ('test POI') then it will be handled correctly.
Is there a way to force POI to handle references correctly?