apt-get remove packagename
will remove the binaries, but not the configuration or data files of the package packagename. It will also leave dependencies installed with it on installation time untouched.
apt-get remove --purge packagename
will remove about everything regarding the package packagename, but not the dependencies installed with it on installation. Both commands are equivalent.