Since you're adding a Xilinx generated core to your design in ISE, you need to add both the VHD file and the NGC file via "Add Source" via the Project menu.
Even easier, depending on how large your ROM needs to be and what data goes into it, would be to not even bother with a Xilinx core, but to use pure VHDL to declare a constant array and initialization values right in your VHDL file. Here is an example:
type array_ROM is array (0 to NUMBER_OF_ROWS-1) of std_logic_vector (ROM_BITWIDTH-1 downto 0);
signal my_ROM : array_ROM
:=
(
x"12345678",
x"ABCDEF01",
...
x"01010101"
);
Now, you don't put the elipsis (...) in that initialization list, just put rows of constants with bit widths matching ROM_BITWIDTH. The NUMBER_OF_ROWS is the number of address locations you need in the ROM. In this example, ROM_BITWIDTH would have to be set to 32 as I've used 32-bit hexadecimal constants in the initialization list. Being a signal, it's actually modifiable, so if you need it to be constant, just use "constant" instead of signal.