CQ5 forms are submited into actions. In order to submit your form into a custom action you need to create an action resource type:
-Create a sling:Folder with a prop sling:resourceType= foundation/components/form/action
Then you create a script or servlet for that resourceType that respond to POST methods and the post selector. This can be a jsp post.Post.jsp or a serlvlet (method=POST, selectors=post, extension=html).
To create a custom validation, you create a clientvalidation.jsp file under that same resourceType.
After that, you can edit your form and select your action.
Adobe documentation about it is pretty straighforward