Boost.Asio is really a good one. The problem is that its documentation is too complex and arranged erratically. If you just need to do the simplest serial port programming, you don't need to use all the advanced features.
Example usage of blocking I/O on serial port.
static boost::asio::io_service ios;
boost::asio::serial_port sp(ios, "/dev/ttyS2");
sp.set_option(boost::asio::serial_port::baud_rate(115200));
// You can set other options using similar syntax
char tmp[64];
auto length = sp.read_some(boost::asio::buffer(tmp));
// process the info received
std::string message = "hello, world";
sp.write_some(boost::asio::buffer(message));
sp.close();