You MUST have at least 3 members in the replicaset, having 2 members is worse than having just a single node (NEVER EVER DO THIS!!). Your problem is worst than you think, the main problem is that if your secondary goes down, your primary will demote itself to a secondary and also effectively go down because you have no more primary. I don't know why you 'have to stick with this configuration, but it can not be a solution.
The solution to this is to create an arbiter, node, this is simply a lightweight member of the replicaset that has no actual data and is simply there to take part in the election for a new primary if one of the other members goes down.