In real-world applications, a GPS system cannot give you a precision/accuracy much better than 5 - 10 meters, in particular if you use a smartphone or a tablet as a receiver. Have a look at the Wikipedia pages regarding GPS for a detailed explanation:
https://en.wikipedia.org/wiki/Gps
https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_Positioning_System
https://en.wikipedia.org/wiki/GPS_augmentation
There isn't any way to improve this accuracy using filtering or other mathematical "data massage" only. You have to rely on some auxiliary positioning system (that is: an auxiliary source of geographic reference data).
The most widely used auxiliary positioning system is the "network" one. It uses both cellular phone antennas and wi-fi antennas as auxiliary reference points. This approach is already used by Google Maps and other services, as well as by Android, Apple and Windows 8 smartphones and tablets. You just have to select it amongst the available location providers in your code.
Wi-Fi/Cell-based systems rarely can give you a precision better that a few meters ("room-level
accuracy"), even if you apply a massive amount of data processing/filtering.
If you want to use a different system, you can evaluate inertial navigation systems or geomagnetic navigation systems (like Indoor Atlas: http://www.indooratlas.com/).
Writing an inertial navigation system (INS) is not easy. You can find some description on the web but most likely you will have to write all of the code by yourself. INS usually cannot give you a precision better than GPS.
Alternatively, you can buy a commercial, ready-to-use geomagnetic system like Indoor Atlas. Geomagnetic systems can be very precise. Indoor Atlas claims a precision of few centimeters.
If you are looking for a wi-fi/cell-antennas-based system, alternative to Google Maps, you can have a look at Combain:
http://combain.com/
http://location-api.com/
I had a very good experience with WiFiSLAM, as well:
http://wifislam.com/