You are not talking about a scenario where CodeExpression is ever useful. It is a source code generator, the specific kind of source code that gets generated is determined by the provider you selected.
But at no point does the user of your project actually care about that language in your envisioned usage. He never looks at it, he never compiles it himself. Only you care, you have to pick the right CodeCompiler. The user only picks the assembly it produces. And by .NET conventions, the language that was used to create an assembly never matters. The metadata inside the assembly is entirely language-agnostic.
CodeObjects are useful in a scenario where source code is automatically generated and added to the user's project. To be compiled, later, when the user builds his project. Good examples are the various designers built into Visual Studio, like the Resource designer and the Winforms designer. By necessity, they must generate code that matches the user's project type.
It should strongly be avoided if you don't need it. Biggest hangup with the CodeDom code generators is that it is only capable of generating a subset of the statements that are valid in a language. And of course ugly to use, it litters your code. You only need to generate text, the language you pick doesn't matter. Since you seem to favor C#, that's the kind of text you ought to generate.
Do consider the bigger solution. It probably would work a lot better if the user can in fact open a custom designer inside Visual Studio itself. So this extra step, running that translator to go from TinyType to an assembly isn't needed anymore. Easier to use, lights-up IntelliSense as well. Now it does make sense to generate code. This is going to take a lot of work, creating designers isn't that simple. Do keep your eyes on the ball, no user is going to enjoy generating that XML file you need. Creating tiny types in C# is already easy, it just doesn't need much help. Either way, the user composing his own types from your tiny types just doesn't need any help either. VS already supports it directly.