handling None values in conversion to numpy recarray

Question

Is there a graceful way of handling None values in a conversion of a list of tuples to a numpy recarray using the single call to np.rec.fromrecords? Assuming I know what I want the missing value to be (e.g. -1 for integers), how do I catch and handle the below contrived example:

import numpy as np
a = [('Bob', 40, 3.14), ('Sue', 38, 6.28), ('Jim', None, 9.42)]
dtype = [('NAME', 'S10'), ('AGE', np.int32), ('SCORE', np.float64)]
try:
    b = np.rec.fromrecords(a, dtype=dtype)
except TypeError:
    # Convert None to 0 for AGE field here instead of raising the error
    raise TypeError('Caught a TypeError')

I'm guessing that I would have to do this on a per-field basis in order to avoid missing true TypeErrors elsewhere in the recarray. Is there any way of isolating where (ie. what fields) in the recarray I want this conversion to apply. My real use case is converting pyodbc records to numpy recarrays.

Answer 1

Return -1 for the NULL column values using the database query, something like this:

SELECT COALESCE(ColumnName, -1) FROM Schema.Table;

This will return -1 for ColumnName values that are NULL, otherwise the actual value is returned. Documentation for COALESCE here if needed. This allows you to provide a NULL replacement value only for the columns you require, and will not mask TypeError exceptions that you should care about.