OrderedMap or List for Redux structure?
https://softwareengineering.stackexchange.com/questions/375770
-
07-02-2021 - |
Question
I'm implementing a Redux store for a React app using redux-saga for API calls. I've used Immutable.js before for other Redux-using React apps but the nature of previous projects always evidently required a normalized structure.
In this new app, the flow is less app-like and more a series of steps. There is an API call to acquire an array of (somewhat sparse) objects, expected to be about 400-600 in total. And then there is an API call per object to pick an analysis out of a very large ML-derived dataset. There is no 'bulk process' API available in the immediate term for the ML dataset (just promised in the future - their engineering resources are still focused on correctness/validity in their dataset).
I'm wondering whether to keep the initial sparse objects as an OrderedMap or a List. Essentially I need to iterate over the objects sequentially, updating each one with data acquired from the ML-dataset via a specific API call. With all objects updated, it's then delivered at once via a user download.
I'm expecting redux-saga to drive the API calls and just spit out actions to keep a progress bar on the UI updated.
This also begs the question: would you ever use an ImmutableJS List as an overall structure for Redux?
Solution
I went ahead and implemented this using redux-saga to run three workers to pull the required urls out of a channel.
export function* locationsWatcher() {
yield takeLatest(FIND_ALL_LOCATIONS_START, handleAllLocations);
}
function* handleAllLocations() {
try {
// create channel to queue induvidual location requests on
const locationChan = yield call(channel);
yield fork(watchLocationRequests, locationChan);
// get Immutable List of base customer ids
const customers = yield select(getCustomersList);
const custArr = customers.toArray();
// put payloads into location channel
for (let i = 0; i < custArr.length; i++) {
const id = custArr[i].id;
const payload = { path: `/find/${id}` }
yield put(locationChan, payload);
}
// put 'end' into channel for last worker to identify finish
yield put(locationChan, 'complete')
} catch (error) {
console.warn('handleAllLocations setup failed')
}
}
function* watchLocationRequests(locationChan) {
// create 3 worker threads
for (var i = 0; i < 3; i++) {
yield fork(handleLocationRequest, locationChan);
}
while (true) {
const { payload } = yield take(FIND_LOCATION_REQUEST);
yield put(locationChan, payload);
}
}
// get a single location
function* handleLocationRequest(locationChan) {
while (true) {
const payload = yield take(locationChan)
if (payload === 'complete') {
yield put(findLocationsEnd());
continue;
}
try {
const response = yield call(request, payload.path);
yield put(findLocationSuccess(payload, response.properties));
} catch (err) {
// update store with 'cannot find' message
yield put(findLocationFailure(payload));
}
}
}
On writing the initial code, I realised I needed to turn the 'loading' in the UI off on completion. So last payload in the channel is a simple string. The channel handler looks out for that to send the 'ending' action. There may be a more elegant way to do that.
I've tested this and for about 250 calls, it takes about 15s (which includes a lot of UI updating) at usually < 300ms response time. Given that there's a progress bar for the user, I think this is a good first cut.
In answer to my question, I don't think there is inherently anything wrong with using an Immutable List and as demonstrated, toArray
will simply provide a JS array.
This pattern could be appropriate for any large number of calls required but a slow api response could impact that dramatically.