Net HTTP unescapes characters?

Question

Does Ruby Net HTTP unescape characters (for example 龅 to 龅)? I send a request using HTTParty with 龅 but the request parameters on the request server show 龅 instead.

Answer 1

Net::HTTP doesn't do any conversion or decoding. It's just a pipe to move data.

If you are seeing converted characters on the server side then something there is changing them before you can see them.

This shows what Net::HTTP is sending. In test.rb:

#!/usr/bin/env ruby
require 'ap'
require 'sinatra'

post '/' do
  t = %w[text/css text/html application/javascript]
  request.body.rewind  # in case someone already read it
  puts "request.accept             #{ request.accept             }"  # ['text/html', '*/*']
  puts "request.accept? 'text/xml' #{ request.accept? 'text/xml' }"  # true
  puts "request.preferred_type(t)  #{ request.preferred_type(t)  }"  # 'text/html'
  puts "request.body               #{ request.body               }"  # request body sent by the client (see below)
  puts "request.scheme             #{ request.scheme             }"  # "http"
  puts "request.script_name        #{ request.script_name        }"  # "/example"
  puts "request.path_info          #{ request.path_info          }"  # "/foo"
  puts "request.port               #{ request.port               }"  # 80
  puts "request.request_method     #{ request.request_method     }"  # "GET"
  puts "request.query_string       #{ request.query_string       }"  # ""
  puts "request.content_length     #{ request.content_length     }"  # length of request.body
  puts "request.media_type         #{ request.media_type         }"  # media type of request.body
  puts "request.host               #{ request.host               }"  # "example.com"
  puts "request.get?               #{ request.get?               }"  # true (similar methods for other verbs)
  puts "request.form_data?         #{ request.form_data?         }"  # false
  puts "request['SOME_HEADER']     #{ request['SOME_HEADER']     }"  # value of SOME_HEADER header
  puts "request.referrer           #{ request.referrer           }"  # the referrer of the client or '/'
  puts "request.user_agent         #{ request.user_agent         }"  # user agent (used by :agent condition)
  puts "request.cookies            #{ request.cookies            }"  # hash of browser cookies
  puts "request.xhr?               #{ request.xhr?               }"  # is this an ajax request?
  puts "request.url                #{ request.url                }"  # "http://example.com/example/foo"
  puts "request.path               #{ request.path               }"  # "/example/foo"
  puts "request.ip                 #{ request.ip                 }"  # client IP address
  puts "request.secure?            #{ request.secure?            }"  # false (would be true over ssl)
  puts "request.forwarded?         #{ request.forwarded?         }"  # true (if running behind a reverse proxy)

  ap request.env["rack.request.form_hash"]  # raw env hash handed in by Rack 
end

In test2.rb:

#!/usr/bin/env ruby

require 'net/http'

uri = URI('http://0.0.0.0:4567/')
res = Net::HTTP.post_form(uri, 'q' => '龅')

Save those to the respective files, then run the first:

ruby test.rb 
== Sinatra/1.3.2 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.3.1 codename Triple Espresso)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop

Followed by the second. You should see something like:

request.accept             ["*/*"]
request.accept? 'text/xml' text/xml
request.preferred_type(t)  text/css
request.body               #<StringIO:0x000001017187c8>
request.scheme             http
request.script_name        
request.path_info          /
request.port               4567
request.request_method     POST
request.query_string       
request.content_length     16
request.media_type         application/x-www-form-urlencoded
request.host               0.0.0.0
request.get?               false
request.form_data?         true
request['SOME_HEADER']     
request.referrer           
request.user_agent         Ruby
request.cookies            {}
request.xhr?               false
request.url                http://0.0.0.0:4567/
request.path               /
request.ip                 127.0.0.1
request.secure?            false
request.forwarded?         false
{
    "q" => "&#40837;"
}

Notice that last line. That's what Net::HTTP sent as the q parameter to the POST request.