Ok, I could finally track down my issue: I still hadn't understand how EXACTLY a ternary tree works at all. Because of that, I didn't visit the left and right nodes when an wildcard could be somewhere inside these branches (sadly that's almost everytime). So my final algorithm is MUCH slower than a search in an ternary tree should be at all (I'm afraid the complexity would be probably something like O(n)), but at least I got it to work.
So it seems this data structure isn't suitable for my needs (searching for wildcard strings) - I would probably get the same speed with an linear list. However, if someone with similiar problems will find this question some day, here's my code (but I don't suggest to use it in real applications, cause it's slow and I guess there are other structures around, that can handle these things much better):
template <class Object>
void TernarySearchTree<Object>::partialMatchSearchInverted(TstTree tree, const char *key)
{
if (!tree) return;
if((tree->splitChar == '*') && ( *key != 0 )){
partialMatchSearchInverted( tree, key+1 ); // wildcard occured, search same node with next key
}
if( *key != 0 ){
if (*key < tree->splitChar){
partialMatchSearchInverted( tree->left, key );
}else if('*' < tree->splitChar){ // a wildcard could be in this branch
partialMatchSearchInverted( tree->left, key );
}
if (*key > tree->splitChar){
partialMatchSearchInverted( tree->right, key );
}else if('*' > tree->splitChar){ // a wildcard could be here too
partialMatchSearchInverted( tree->right, key );
}
}
if ((*key == tree->splitChar) || (tree->splitChar == '*')){
if (*key != 0){
partialMatchSearchInverted( tree->mid, key+1 ); // search next pattern char
}
}
if ( ( *key == 0 ) && ( tree->splitChar == 0 ) ){
pmVectorPtr->add( tree->index );
}
}