I do not think there is any simple or uniform way to do what you are asking. The command lsof]1 manages to determine this information so you may want to take a look at that code to see what they are doing.
Off the top of my head to determine if a descriptor is a socket you can use getsockopt(2). If a call to getsockopt fails and errno == ENOTSOCK then your descriptor is not a socket. likewise you can use isatty(3) to determine if a descriptor belongs to a serial port or terminal.