The easiest way to do this is to use the simple index at PyPI which lists all packages without overhead. You can then request the JSON of each package individually by performing a GET request to the URLs mentioned in your question.
JSON API for PyPi - how to list packages?
Pregunta
There is a JSON API for PyPI which allows getting data for packages:
http://pypi.python.org/pypi/<package_name>/json
http://pypi.python.org/pypi/<package_name>/<version>/json
However, is it possible to get a list of all PyPI packages (or, for example, recent ones) with a GET call?
Solución
Otros consejos
I know that you asked for a way to do this from the JSON API, but you can use the XML-RPC api to get this info very easily, without having to parse HTML.
try:
import xmlrpclib
except ImportError:
import xmlrpc.client as xmlrpclib
client = xmlrpclib.ServerProxy('https://pypi.python.org/pypi')
# get a list of package names
packages = client.list_packages()
I tried this answer, but it's not working on Python 3.6
I found one solution with HTML parsing by using lxml package, But you have to install it via pip command as
pip install lxml
Then, try the following snippet
from lxml import html
import requests
response = requests.get("https://pypi.org/simple/")
tree = html.fromstring(response.content)
package_list = [package for package in tree.xpath('//a/text()')]
NOTE: To make tasks like this simple I've implemented an own Python module. It can be installed using pip
:
pip install jk_pypiorgapi
The module is very simple to use. After instantiating an object representing the API interface you can make use of it:
import jk_pypiorgapi
api = jk_pypiorgapi.PyPiOrgAPI()
n = len(api.listAllPackages())
print("Number of packages on pypi.org:", n)
This module also provides capabilities for downloading information about specific packages as provided by pypi.org
:
import jk_pypiorgapi
import jk_json
api = jk_pypiorgapi.PyPiOrgAPI()
jData = api.getPackageInfoJSON("jk_pypiorgapi")
jk_json.prettyPrint(jData)
This feature might be helpful as well.
Here's Bash one-liner:
curl -sG -H 'Host: pypi.org' -H 'Accept: application/json' https://pypi.org/pypi/numpy/json | awk -F "description\":\"" '{ print $2 }' |cut -d ',' -f 1
# NumPy is a general-purpose array-processing package designed to...