C++ L"whatever" -> wstring -> basic_string<T>

Question

I'm writing a "string splitter" for a parser, and i want to be able to use both char and wchar_t. I have the following method :

static void tokenize(const basic_string<T> &, vector<CToken> &);

And i would like to give it a wide string to parse :

vector<CToken> tTokens;
CTokenizer<wstring>::tokenize(L"if(i == 0) { print i + 2; } else { return; }", tTokens);

I also tried :

vector<CToken> tTokens;
const wstring sCode = L"if(i == 0) { print i + 2; } else { return; }";
CTokenizer<wstring>::tokenize(sCode, tTokens);

So i assume there is something i don't understand about templates. What can i do please ?

Thanks for your help !

EDIT : Here is my build log :

1>------ Build started: Project: c_parser, Configuration: Debug Win32 ------
1>  Parsercpp.cpp
1>c:\users\virus\documents\visual studio 2012\projects\c_parser\c_parser\parsercpp.cpp(44): error C2958: the left parenthesis '(' found at 'c:\users\virus\documents\visual studio 2012\projects\c_parser\c_parser\parsercpp.cpp(38)' was not matched correctly
1>c:\program files (x86)\microsoft visual studio 11.0\vc\include\xstring(529): error C2621: member 'std::_String_val<_Val_types>::_Bxty::_Buf' of union 'std::_String_val<_Val_types>::_Bxty' has copy constructor
1>          with
1>          [
1>              _Val_types=std::_Simple_types<std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t>>>
1>          ]
1>          c:\program files (x86)\microsoft visual studio 11.0\vc\include\xstring(532) : see reference to class template instantiation 'std::_String_val<_Val_types>::_Bxty' being compiled
1>          with
1>          [
1>              _Val_types=std::_Simple_types<std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t>>>
1>          ]
1>          c:\program files (x86)\microsoft visual studio 11.0\vc\include\xstring(627) : see reference to class template instantiation 'std::_String_val<_Val_types>' being compiled
1>          with
1>          [
1>              _Val_types=std::_Simple_types<std::basic_string<wchar_t,std::char_traits<wchar_t>,std::allocator<wchar_t>>>
1>          ]
1>          c:\program files (x86)\microsoft visual studio 11.0\vc\include\xstring(700) : see reference to class template instantiation 'std::_String_alloc<_Al_has_storage,_Alloc_types>' being compiled
1>          with
1>          [
1>              _Al_has_storage=false,
1>              _Alloc_types=std::_String_base_types<std::wstring,std::allocator<std::wstring>>
1>          ]
1>          c:\users\virus\documents\visual studio 2012\projects\c_parser\c_parser\parsercpp.cpp(104) : see reference to class template instantiation 'std::basic_string<_Elem>' being compiled
1>          with
1>          [
1>              _Elem=std::wstring
1>          ]
1>c:\users\virus\documents\visual studio 2012\projects\c_parser\c_parser\parsercpp.cpp(104): error C2664: 'nsParser::CTokenizer<T>::tokenize' : cannot convert parameter 1 from 'const std::wstring' to 'const std::basic_string<_Elem> &'
1>          with
1>          [
1>              T=std::wstring
1>          ]
1>          and
1>          [
1>              _Elem=std::wstring
1>          ]
1>          Reason: cannot convert from 'const std::wstring' to 'const std::basic_string<_Elem>'
1>          with
1>          [
1>              _Elem=std::wstring
1>          ]
1>          No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

EDIT :

template<class T>
class CTokenizer
{
public:
    static vector<T> s_tKeywords;
public:
    static void tokenize(const basic_string<T> &, vector<CToken> &);
};

The code from parser.cpp is the one from above :

vector<CToken> tTokens;
const wstring sCode = L"if(i == 0) { print i + 2; } else { return; }";
CTokenizer<wstring>::tokenize(sCode, tTokens);

Answer 1

You should be instantiating CTokenizer<wchar_t>, not CTokenizer<std::wstring>. The way you're doing it, the signature of tokenize() becomes

static void tokenize(const basic_string<wstring> &, vector<CToken> &);

Answer 2

Either change tokenize's declaration to

static void tokenize(const T &, vector<CToken> &);

or the call to

CTokenizer<wchar_t>::tokenize(L"if(i == 0) { print i + 2; } else { return; }", tTokens);

---

Edit after your comment: Then use the "or" part (as opposed to the "either")...

And since T is then the "char type" (not the string type), I suggest you rename it to something meaningful.

Taking the code of your second EDIT, after those changes the class looks like:

template<class CharT>
class CTokenizer
{
public:
    static vector< basic_string<CharT> > s_tKeywords;
public:
    static void tokenize(const basic_string<CharT> &, vector<CToken> &);
};

then you call tokenize as I wrote above (with wchar_t, not wstring, as the template type parameter). (wstring is just a type-alias for basic_string<wchar_t>.)