RNG: removing bias by counting bits

Question

I have a hardware RNG that sends me some random bits. Due to nature of that RNG, the source is biased. Then, the bitstream is processed in a following way to remove the bias:
two counters are used, which count amounts of 1's and 0's in the stream and when the difference is above the threshold, the next bit out ot the rng is forced to either 0 or 1.
I have concerns about this. This just means that we have directly introduced some dependency of future bits on past bits, which totally destroys randomness, right? Is this a valid concern? If yes, what are proper ways to deal with bias in RNG?