Turns out you have two choices to do this:
1) Add parameters to an exception:
$e = new Exception('I pooped my pants');
$e->color = 'brown';
$->smell = 'bad';
color and smell would then be in the data parameter of the error response.
2) Pass the server (in my code, $jsonRpc) into the object (in my code, it would look something like: new Testi($jsonRpc)), and use the fault(...) method, which allows you to pass the data array/object into it.
The latter approach gives you more flexibility as you can do data->code, data->message, neither of which can you set on the $e object, because they are existing, protected parameters. But you are then coupling your model to the $jsonRpc server, which is not good.
All that being said, it's not the correct way to respond to the scenario I outlined above. The error response is more or less reserved for true, unrecoverable server errors, akin to real exceptions, not user validation errors. In my case it was better to define a response type that allows me to return success/fail values with appropriate response codes. {success: false, code: 5}.
Cheers, Adam