This can be achieved in Page TSconfig:
RTE.default.userElements {
10 = Various elements
10 {
1 = Table
1.description = Wrap in a table
1.content = <table><tr><td>|</td></tr></table>
}
}
To enable the "Insert custom element" button:
RTE.default.showButtons := addToList(user)
RTE.default.hideButtons := removeFromList(user)
Edit: You might need to fiddle with the parser to allow certain tags or attributes. I think this one did it for my case:
RTE.default.proc.HTMLparser_db.allowTags = html, body, div, span, applet, object, iframe, h1, h2, h3, h4, h5, h6, p, blockquote, pre, a, abbr, acronym, address, big, cite, code, del, dfn, em, font, img, ins, kbd, q, s, samp, small, strike, strong, sub, sup, tt, var, dl, dt, dd, ol, ul, li, fieldset, form, label, legend, table, caption, tbody, tfoot, thead, tr, th, td
Source: htmlarea manual and this snippet