Update
As mentioned below, pandas is now available for python 3.3, 3.4 and 3.5 source
For system wide install use:
sudo apt-get install python3-pandas
Original:
If this information http://packages.ubuntu.com/precise/python/ is correct there is no pandas package for Python 3. You can install current pandas using virtualenv:
apt-get install python-virtualenv virtualenvwrapper
mkvirtualenv -p python3 pandas_env
pip install pandas
Generally it is a good idea to create separate virtual environments when working with Python and to avoid manual messing with system wide packages.