Sage does not use the python installed in your system. It uses its own copy of python so while working with sage programs you would have to run the programs from terminal using this command
$ sage --python <Python_File>
Similarly sage does not uses the modules installed in the system's python. The Modules need to be installed under sage libraries. You have downloaded and extracted the folder right but used a wrong command. Now go again in the xlrd
extracted folder and use this command.
$ sage --python setup.py install
Now Everything should work.
Cheers