How should I provide detailed information about an error, if not through HTTP status codes?

Question

This question comes from here. I believe it deserves to be on its own instead of being lost in a long question with multiple points.

I'm building an API and find myself limited by HTTP error codes. They are just:

• Not specific enough (example: HTTP 403 Forbidden to indicate that the price is outside of the allowed range and the same HTTP 403 Forbidden to indicate that the identifier of the product has a wrong format), and at the same time:

• Many of them are useless in my case (example: HTTP 402 Payment Required).

Specifically when exceptions are encountered, can I use my own HTTP response codes? If not, what are my options if I want to provide the client with information more specific than “HTTP 500 Sorry we failed but we won't tell you why”?

Note: I develop the server and the client at the same time, so there would be no other clients accessing the API.

Answer 1

Don't invent status codes

You are not expected to invent your own response codes, since the point of the API is to use a standard interface any developer can understand.

The fact that you maintain both the API and its client is irrelevant: since everyone can trace the calls to the API, everyone can implement a different client. The point of using standard interfaces is also:

• To facilitate maintenance. If your app should later be maintained by your coworker, he can find his way with ease. He will be lost if it appears that HTTP 306 means success, HTTP 500 is a “Not found” and HTTP 909 is “Internal Server Error”, unless the error is a FileNotFound exception, in which case it's HTTP 404.

• To facilitate the work of your system administrator. Tools which deal with server logs use status codes to determine whether this is a error or not. Two concrete examples are:

  1. The displaying of the number of errors (HTTP 4xx and HTTP 5xx) for a given server and:

  2. The tracking of hacking attempts. For instance, when I see a hacking attempt which results in HTTP 200, this immediately attracts my attention: it should be HTTP 4xx instead.

• To avoid messing with your HTTP server. IIS, for instance, can easily be offended by some status codes, will catch them and generate its own responses. Configuring both your app and IIS is not always easy (and I've spent several days banging my head with this issue when I started programming ASP.NET websites).

• To avoid problems with proxies. For instance, if your API is hosted on a server accessed through Nginx, I'm not sure sysadmins will be happy to spend a few hours redoing all the configuration for your app.

  A basic example. When configuring failover, I use proxy_next_upstream directive which indicates in which cases Nginx should switch to the failover machine. Looking at the documentation, I have an impression that you can't set the directive to, say, HTTP 200, so if your API uses HTTP 200 as “Fatal error, all data was corrupted so you may be better using another mirror”, you're out of luck.

• To avoid reinventing the wheel. Why would you waste your time standardizing your set codes, while there is already one?

• To leverage support from many tools. For instance, Fiddler relies on HTTP response code to colorize the items. I'm not sure if it's easy to configure it to use different statuses. In the same way, different API testing tools may rely on status codes as well (while CURL, conveniently, couldn't care less about the response status, Python's requests package, for instance, will do a redirection in a presence of HTTP 3xx code).

By the way, browsers themselves interpret the response code:

• AJAX requests returning HTTP 3xx may lead to automatic redirection.

• HTTP 4xx and HTTP 5xx are displayed in red in browsers' developer tools.

Provide additional information

The response code other than HTTP 200 doesn't mean that you can't feed the client with JSON as well (unless it's HTTP 204, in which case there should be no content). This means that you're free to give as much information as you want to the client through JSON. Personally, I include:

• The ID of the error in a form of a string. For instance price-range-invalid or product-not-found.

• The URI containing additional help, when relevant (when the API is large enough to contain individual pages for every error message). For instance http://example.com/api/v1/errors/price-range-invalid.

• The description. For instance “The current price is outside the allowed range. It should be superior to 0 and inferior or equal to 50000.”.

See also how other APIs handle the errors and what sort of information they provide. For instance, this is how Google+ API errors look like:

HTTP 403

{
 "error": {
  "errors": [
   {
    "domain": "usageLimits",
    "reason": "ipRefererBlocked",
    "message": "There is a per-IP or per-Referer restriction configured on your API key and the request does not match these restrictions. Please use the Google Developers Console to update your API key configuration if request from this IP or referer should be allowed.",
    "extendedHelp": "https://console.developers.google.com"
   }
  ],
  "code": 403,
  "message": "There is a per-IP or per-Referer restriction configured on your API key and the request does not match these restrictions. Please use the Google Developers Console to update your API key configuration if request from this IP or referer should be allowed."
 }
}

Don't provide too much information

Finally, make sure you don't include the exception itself in the details, since it may contain sensitive information and represent a path for a hacker to explore (including the stack trace).

For instance, some PHP websites, when encountering a database related error, just show the SQL query and the error to the user. For a hacker, this is very convenient. For a legitimate user, this is just unfriendly and unhelpful. Don't do that.

Answer 2

The only status codes you need from HTTP are 200 for Success and 500 for failure, so that you know whether you are having a successful conversation or not. (Perhaps also 404 if your API is not the only application running on your server.)

Once a successful conversation has been established, then any other error that might happen has absolutely nothing to do with the communication protocol; it is an error which is specific to your application, so it should be communicated within your XML or JSON or whatever it is that you are using to implement your API.

Always keep this question in mind:

What if you were communicating over a medium other than HTTP?