You just need an extra double quote to escape it (you don't use \
to escape a string in vb):
Dim arrayWithQuote() As Char = {CChar("a"), CChar("b"), CChar("""")}
As Tim Pointed out you should use char literals to do this sort of thing:
Dim arrayWithQuote() As Char = {"a"c, "b"c, """"c}
Alternatively you could do this:
Dim arrayWithQuote() As Char = {"a"c, "b"c, Microsoft.VisualBasic.Chr(34)}