SFML is a C++ library, thus every time you switch the compiler, you need to recompile the library. The pre-compiled binaries provided by SFML are for the older version of TDM and thus not compatible with TDM 4.8.1. Building libraries yourself will become more important the longer you develop, so the sooner you start the better, plus it's not that hard.
As a personal side note: While I appreciate the hard work by TDragon, it's quite a no-go that the TDM compilers change the default linking behavior of GCC, thus breaking about every build script in existance, unless it's specially modified for TDM. Unfortunately the Code::Blocks community doesn't care at all. Thus I feel the need to point people to MinGW Builds whenever possible.