Each VTK source has a dataset for both scalars and vectors.
The trick I use in my program to getting the color and size to differ is to bypass the mayavi source and directly in the VTK source, use scalars for color and vectors for size (it probably works the other way around as well).
nodes = points3d(x,y,z)
nodes.glyph.scale_mode = 'scale_by_vector'
#this sets the vectors to be a 3x5000 vector showing some random scalars
nodes.mlab_source.dataset.point_data.vectors = np.tile( np.random.random((5000,)), (3,1))
nodes.mlab_source.dataset.point_data.scalars = np.random.random((5000,))
You may need to transpose the 5000x3 vector data or otherwise shift the matrix dimensions somehow.