I ended up using the jquery built in ajax handler and then did a reload on the jtable as shown below:
on the jtable customfield definition I call my own refreshTable function:
display: function (data) {
if (data.record) {
return '<button title="Refresh'+data.record.table+'" class="jtable-command-button jtable-edit-command-button" onclick="refreshTable(data.record.table); return false;"><span>Refresh '+data.record.table+'</span></button>';
}
I then handle the ajax request with jquery to do some serverside mojo and then if success, reload the jtable:
function refreshTable(table) {
$.post("/myurlpath/refreshTable", "table="+table,
function(results)
{
$('#MyTableDiv').jtable('reload');
}
, "json");
}
I left off the ajax error handling for code readability in the post, but this handles what I needed (to get a call out to the server from a custom jtable button).