The Jess function you want is
(mod <numeric-expression> <numeric-expression>)
As for learning Jess, there Earnest Friedman-Hill's book (not on the internet, AFAIK) is still the best, I hear. But the manual that comes with Jess is very good, and you can learn a lot from that.
Since Jess is a Lisp dialect, you can learn from any tutorial on Lisp. Then, browse the reference, just to get an idea what is available.