I/O ports are not memory-addressed. To put this another way, x86 I/O ports have an entirely separate address space than memory. There is no overlap with memory. A different set of CPU instructions ("IN", "OUT") are used when operating on I/O ports. The I/O ports are "discovered" by device drivers and then registered/reserved within the kernel using "request_region"; see, for example, serial8250_request_std_resource() in the kernel source.
Don't confuse this with "memory-mapped I/O" where I/O regions are mapped into the processor's physical memory space. These are registered/reserved with request_mem_region. You can view the latter with cat /proc/iomem
.
Serial devices can be devised to work in either the I/O space or as memory-mapped I/O so, for you, which way to do it will be very dependent on your specific device.