Serving heterogenous data from backend

Question

There is an application which serves a huge list of resources and in the UI the users are able to filter the resources by type and other properties.

The resources are of different types but they have few properties common in them. For example there are resources of type Car, Book and Pet each has colour, price, owner and type as the common property.

At present the application has an endpoint /resources which sends all the resources at once as JSON which is a list of resource object, now due to this approach nearly all the objects have few fields with null values and the properties which doesn't makes sense at all.

Lets take an example:

[{
  type: 'car',
  color: 'red',
  price: '1000',
  owner: 'foo',
  engines: '2',
  tyres: '4',
  capacity: '2',
  name: null
},{
  type: 'pet',
  color: 'red',
  price: '10',
  owner: 'foo',
  engines: null,
  tyres: null,
  capacity: null,
  name: 'koo'
},{
  type: 'book',
  color: null,
  price: '1000',
  owner: 'foo',
  engines: null,
  tyres: null,
  capacity: null,
  name: 'theory of everything'
}, ...];

Once data is in front end the user can apply all kind of filtering to get the required information.

Problems:

1. There is a single UI table and the header is changed based on the resource shown.
2. There is a lot of checking done in the code base in front end(typescript) and back-end(java) and code based seems(is) repetitive and fragile.

Questions:

The project is still in early phase and hence I can ask for refactoring which can provide long term gain I was thinking if there needs to have different end points and then aggregate them in the front end? but then there would be many api calls for each resource type and UI can be in inconsistent state if all data fails to load.

Answer 1

An application can provide a data stream of objects of various types, there is nothing wrong with this. But if consumer applications shall stay robust and maintainable, it is a good idea to make sure each of the objects carries enough metadata to let the consumer of that stream interpret the data correctly in a more or less general fashion, without lots of error-prone implicit assumptions about the data structure or its semantics.

For example, if the only purpose of your idea of different endpoints is to pre-filter the objects by type, but each of the objects also carries an explicit type information as shown in your example, then different endpoints won't bring you any benefit in terms of maintainability, because the consumer could just filter the object into different streams utilizing the type field, if required. The type information is the important piece of metadata here.

Or, if the main use case here is to let the user filter the information provided from that data stream, the attribute/column names metadata information is crucial, and that is provided by the JSON format, too. If, however, the data should also be sorted in the UI, one needs to know the exact data type of the attributes, because numbers are sorted differently when interpreted as strings. These could be provided either by some separate JSON-Schema, or by making the value literals type-aware (opposed to your example above, where each number seems to be encoded in a string).