LWP::UserAgent set ip of requested url so LWP doesn't have to do dns lookup
-
02-07-2021 - |
Вопрос
I'm using LWP::UserAgent to request a lot of page content. I already know the ip of the urls I am requesting so I'd like to be able to specify the ip address of where the url I am requesting is hosted, so that LWP does not have to spend time doing a dns lookup. I've looked through the documentation but haven't found any solutions. Does anyone know of a way to do this? Thanks!
Решение
So I found a module that does exactly what I'm looking for: LWP::UserAgent::DNS::Hosts
Here is an example script that I tested and does what I specified in my question:
#!/usr/bin/perl
use strict;
use LWP::UserAgent;
use LWP::UserAgent::DNS::Hosts;
LWP::UserAgent::DNS::Hosts->register_host(
'www.cpan.org' => '199.15.176.140',
);
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;
#actually enforces new DNS settings as if they were in /etc/hosts
LWP::UserAgent::DNS::Hosts->enable_override;
my $response = $ua->get('http://www.cpan.org/');
if ($response->is_success) {
print $response->decoded_content; # or whatever
}
else {
die $response->status_line;
}
Другие советы
Hum, your system should already be caching DNS responses. Are you sure this optimisation would help?
Option 1.
Use
http://192.0.43.10/
instead of
http://www.example.org/
Of course, that will fail if the server does name-based virtual hosting.
Option 2.
Replace Socket::inet_aton
(called from IO::Socket::INET called from LWP::Protocol::http) with a caching version.
use Socket qw( );
BEGIN {
my $original = \&Socket::inet_aton;
my %cache;
my $caching = sub {
return $cache{$_[0]} //= $original->($_[0]);
};
no warnings 'redefine';
*Socket::inet_aton = $caching;
}
Simply replace the domain name with the IP address in your URL:
use strict;
require LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;
# my $response = $ua->get('http://stackoverflow.com/');
my $response = $ua->get('http://64.34.119.12/');
if ($response->is_success) {
print $response->decoded_content; # or whatever
}
else {
die $response->status_line;
}