HTTP request not going through proxy

Question

I have written this code to fire a http request through a proxy. But the request does not seem to use proxy. Even though I give a wrong proxy, it is returning OK.

Is there any way I can check, whether the HTTP request went via proxy? What is the issue in this code which makes it not use proxy?

sub fire_http_request_through_proxy()
{
    my $proxy = $_;
    my $ua = LWP::UserAgent->new;
    $ENV{HTTP_PROXY} = $proxy;
    $ua->env_proxy; # initialize from environment variables
    $ua->timeout(20);
    my $response = $ua->get('http://www.google.com');
    delete $ENV{HTTP_PROXY};
    if ($response->is_success)
    {
       print $response->decoded_content . "\n";
    }
    else
    {
       die $response->status_line;
    }  
}

Answer 1

Are you sure that $_ has a true value? This dies appropriately for me:

#!/usr/bin/env perl

use strict;
use warnings;
use LWP::UserAgent;

fire_http_request_through_proxy();

sub fire_http_request_through_proxy {
    my $ua = LWP::UserAgent->new;
    local $ENV{HTTP_PROXY} = 'http://1.1.1.1';
    $ua->env_proxy;    # initialize from environment variables
    $ua->timeout( 20 );
    my $response = $ua->get( 'http://www.google.com' );
    delete $ENV{HTTP_PROXY};
    if ( $response->is_success ) { 
        print $response->decoded_content . "\n";
    }   
    else {
        die $response->status_line;
    }   
} 

So, maybe $_ isn't what you think it is. If it's not defined, then no proxy will be used. Having said that, $_ is probably not the variable you want to use here. You could either declare a variable for use in this case, pass a variable right to the subroutine or actually set an ENV variable outside of the script.

One other point. Rather than setting and deleting the ENV var in your script, just declare the change with local and it will only take effect inside this block of code. That way you don't have to clean up after yourself and you don't risk overriding vars which may have been set elsewhere.

Answer 2

Sebastian and oalders have already solved your problem, but I'd just like to note that you don't need to mess around with $ENV{HTTP_PROXY} anyway — you can just use $ua->proxy(), like this:

$ua->proxy( http => 'http://1.1.1.1' );

or even:

$ua->proxy( ['http', 'https', 'ftp'] => 'http://1.1.1.1' );

---

Ps. If you really want to check which proxy was used by LWP for a particular request, you can peek at $response->request->{proxy}, which should be a URI object. However, as far as I know, this property is undocumented (I found out about it by reading the source) and thus subject to change in later versions of LWP. Use at your own risk!

Answer 3

Take a look at your code sub fire_http_request_through_proxy(), especially the last two characters... This is a prototype. Basically you are saying "I don't take any arguments during compile-time".

I guess you are simply invoking the method before its declaration -> Always use warnings:

main::fire_http_request_through_proxy() called too early to check prototype at test.pl line ...

So either change it to fire_http_request_through_proxy or change it to fire_http_request_through_proxy($) and invoke it after its declaration.

More about prototyping in perlsub.

Answer 4

Be sure

• to read parameter as $_[0] or pop, not $_
• to not include () in sub definition

Script:

sub fire_http_request_through_proxy {
  my $proxy = $_[0];
  my $timeout = 20;
  my $url = 'http://www.google.com';
  my $ua = LWP::UserAgent->new;
  $ua->proxy(['http', 'https', 'ftp'] => $proxy);
  $ua->timeout($timeout);
  my $response = $ua->get($url);
  if ($response->is_success) {
    print $response->decoded_content . "\n";
  }
  else {
    die $response->status_line;
  }  
}

Test:

To make it work, parameter of proxy has to be in correct format (http://host:port)

fire_http_request_through_proxy('http://176.34.248.142:9050');