converting legacy data to json format

Question

I am working with some legacy code where I am using a column value and its earlier history to make some comparisons and highlight the differences. However, the column is stored in an arbitary delimited fashion and the code around comparison is well, very hard to understand.

My initial thought was to refactor the code - but when I later thought about it, I thought why not fix the original source of the issue which is the data in this case. I have some limitations around the table structure and hence converting this from a single column into multiple column data is not an option.

With that, I thought if its worth to convert this arbitary delimited data into a standardized format like json. My approach would be to export this data to some file and apply some regular expressions and convert this data to json format and then re-import it back.

I wanted to check with this group if I am approaching this problem the right way and if there are other ideas that I should consider. Is JSON the right format in the first place? I'd like to know how you approached a similar problem. Thanks!

Answer 1

However, the column is stored in an arbitary delimited fashion and the code around comparison is well, very hard to understand.

So it seems the problem is in the maintainability/evolvability of the code for adding or modifying functionalities.

My initial thought was to refactor the code - but when I later thought about it, I thought why not fix the original source of the issue which is the data in this case.

My opinion is: why? I think your initial thought was right. You should refactor (i.e. " ...restructuring an existing body of code, altering its internal structure without changing its external behavior") to gain what you want. If you change the data format you could end with a messy code above the new cooler data format in any case.

In others words you can have all the combinations because they are not strictly related: you can have a cool data format with a messy cody or with a fantastic code, and you can have a ugly data format with a fantastic code or a messy code.

It seems you are in the last case but moving from this case to: cool data format + messy code and then to cool data format + fantastic code is less straightforward than: fantastic code + ugly data format and then cool data format + fantastic code (optional).

So my opinion is point your goal in a straightforward way:

1) write tests (if you haven't) for the current functionalities and verify them with the current code (first step for refactoring)

2) change the code driven by the tests wrote in 1

3) If you still want, change the data format ever guided by your tests

Regards