I want an Indexer but types of the setter and the getter should be different.
You can't do that, basically - not when the two indexers have the same signature.
Even if you could, I'd say it would be a pretty odd design, which would surprise users of your API.
The relevant bit of the C# specification is section 10.9:
The signature of an indexer must differ from the signatures of all other indexers declared in the same class.