For every folder in the user's profile, you'll want to ensure the folder is removed. So, you can adjust your example like so:
<Directory Id='TARGETDIR' Name='SourceDir'>
<Directory Id="ProgramMenuFolder">
<Directory Id="ProgramMenuCompanyDir" Name="Company Name">
<Directory Id="ProgramMenuProgramDir" Name="ProgramName"/>
</Directory>
</Directory>
</Directory>
<Component Id="ProgramMenuCompanyDirComponent" Directory='ProgramMenuCompanyDir'>
<RegistryValue Root="HKCU" Key="Software\[Manufacturer]\[ProductName]" Name='InstallVersion'
Value="[ProductVersion]" Type="string" />
<RemoveFolder Id="RemoveProgramMenuCompanyDir" On="uninstall" />
</Component>
<Component Id="ProgramMenuProgramDirComponent" Directory='ProgramMenuProgramDir'>
<RegistryValue Root="HKCU" Key="Software\[Manufacturer]\[ProductName]" Name='InstallFolder'
Value='[INSTALLFOLDER]' Type="string" />
<RemoveFolder Id="RemoveProgramMenuProgramDir" On="uninstall" />
</Component>
That ensures that all the per-user folders have per-user key paths (the registry keys) and are correctly marked to be removed during uninstall.