Extract fan database from Facebook page

Question

I am trying to export Facebook Page Fans.

The closer I found was this article.

It states:

Getting fans from a Facebook page is not yet supported by the Facebook API. Luckily, the Facebook Web interface uses a simple AJAX/JSON call to supply the data when you view the page.the data when you view the page.

And he explains what he does like this:

My strategy to set this data free was to sniff the network traffic with the Wireshark tool, then replay the HTTP calls with a ruby script.calls with a ruby script.

I don´t know anything about ruby so I started trying with a PHP scripts left in one of the comments, the one by: "Etienne Bley"

The script goes like this.

The script says you can download Charles Proxy to find this variables:

$cookie
$node_id
$post_form_id
$fb_dtsg

When I use the Charles Proxy Soft, and login to as administrator I get this:

And from there I get what I guess is the cookie variable:

BTW Is it safe to share the whole cookie?? is it helpful? (if it is I´ll edit asap)

The script also says:

// set settings in these 4 lines from results of charles when getting the 2nd page of "Get All Fans" in FB ( you need to be admin of fan page to do this )

I can´t understand what does he mean by: getting the 2nd page

So, my questions:

1) What are this variables?

2) What are their values? How should/can I get them?

3) To have this scripts set correctly is the only thing I need for this script to work?

I hope the question is clear enough, if not please ask any questions you need!

Thanks in advance!

Answer 1

I don't know about Charles Proxy Soft, but I used Chrome's excellent Inspector to trace the request.

Steps:

1. Use Chrome to navigate to the Facebook Page you're interested in
2. Open up the Inspector (CTRL+Shift+J on Windows), go to the "Resources" tab and "Enable Resource Tracking".
3. On the Facebook page, click "See all" in the Fans box on the left side of the page.
4. Scroll to the bottom of the fan list, and click "Next"
5. In the Resources tab, you'll have a request to /ajax/social_graph/fetch.php. Click on that, and in the Headers tab you'll see what you need. In my example:

I'm sure you can do that with a hundred different other programs, I find it easier to use Chrome since it's already there :)

Answer 2

Alright, so it seems this is all simple. I recommend getting a copy of Fiddler to inspect this plan yourself.

I opened up a fan page, went to view the fans, and hit next page. I saw a POST request for http://www.facebook.com/ajax/social_graph/fetch.php?__a=1. What I got back was a really nice JSON array, containing all of the fans.

If we inspect the variables posted, it becomes obvious...

edge_type = fan
page = 1
limit = 100
node_id = 123123123123123123123 (ID of the fan page I'm assuming)
class = FanManager
post_form_id = 97823498723498 (No idea, but I bet you can get this from the dialog)
fb_dtsg = a1s3d5f (No idea)
lsd = 
post_form_id_source = AsyncRequest

Anyway, what you are interested in is page and limit. I bet if you set page to 0 and limit to 500 or whatever, you will get what you are looking for. In the event you can't change limit reliably, just leave it at 100 and keep incrementing page. Also, I have my cookies in there, with the session information. How you will get those and post from PHP I don't know, but I hope this gives you some things to go on.

Again, get Fiddler, inspect what happens when you browse the page.