LWP::UserAgent set ip of requested url so LWP doesn't have to do dns lookup

Question

I'm using LWP::UserAgent to request a lot of page content. I already know the ip of the urls I am requesting so I'd like to be able to specify the ip address of where the url I am requesting is hosted, so that LWP does not have to spend time doing a dns lookup. I've looked through the documentation but haven't found any solutions. Does anyone know of a way to do this? Thanks!

Answer 1

So I found a module that does exactly what I'm looking for: LWP::UserAgent::DNS::Hosts

Here is an example script that I tested and does what I specified in my question:

#!/usr/bin/perl 
use strict;
use LWP::UserAgent;
use LWP::UserAgent::DNS::Hosts;

LWP::UserAgent::DNS::Hosts->register_host(
        'www.cpan.org' => '199.15.176.140',
);

my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;

#actually enforces new DNS settings as if they were in /etc/hosts
LWP::UserAgent::DNS::Hosts->enable_override;

my $response = $ua->get('http://www.cpan.org/');

if ($response->is_success) {
    print $response->decoded_content;  # or whatever
}
else {
    die $response->status_line;
}

Answer 2

Hum, your system should already be caching DNS responses. Are you sure this optimisation would help?

---

Option 1.

Use

http://192.0.43.10/

instead of

http://www.example.org/

Of course, that will fail if the server does name-based virtual hosting.

---

Option 2.

Replace Socket::inet_aton (called from IO::Socket::INET called from LWP::Protocol::http) with a caching version.

use Socket qw( );
BEGIN {
    my $original = \&Socket::inet_aton;

    my %cache;
    my $caching = sub {
       return $cache{$_[0]} //= $original->($_[0]);
    };

    no warnings 'redefine';
    *Socket::inet_aton = $caching;
}

Answer 3

Simply replace the domain name with the IP address in your URL:

use strict;
require LWP::UserAgent;

my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;

# my $response = $ua->get('http://stackoverflow.com/');
my $response = $ua->get('http://64.34.119.12/');

if ($response->is_success) {
    print $response->decoded_content;  # or whatever
}
else {
    die $response->status_line;
}