using GNU Parallel for pagination

Question

I like GNU Parallel and have tried to use it for pagination but need help to get it working successfully. Basically, I am following the use cases on the Quickblox API guide to get data:

http://quickblox.com/developers/Custom_Objects#Get_related_records

The maximum number of records one can retrieve is 100 per page, and one can only retrieve a page at a time. These are specified via the -d parameter. I want to use GNU Parallel to obtain pages 1..79.

I found a thread that explains how to use GNU Parallel when you have parameters that take on many different values but haven't been able to successfully adapt it to my case.

GNU Parallel - parallelize serial command line programs without changing them

Your help would be greatly appreciated!

curl -X GET -H "QB-Token: 7de49c25f44e557aeed1b635" -d "page=3" -d "per_page=100" https://api.quickblox.com/users.xml > qblox_users_page3_100perpage

Answer 1

If you want output in different files:

parallel 'curl -X GET -H "QB-Token: 7de49c25f44e557aeed1b635" -d "page={}" -d "per_page=100" https://api.quickblox.com/users.xml > qblox_users_page{}_100perpage' ::: {1..79}

If you want it in a single big file:

parallel -k 'curl -X GET -H "QB-Token: 7de49c25f44e557aeed1b635" -d "page={}" -d "per_page=100" https://api.quickblox.com/users.xml' ::: {1..79} > qblox_users