Design Pattern For "Load Format Save Data" [closed]

Question

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.

Closed 9 years ago.

I need to acomplish things like loading data from xls source (ole db), formatting it according to specification of output file, merging proccessed fields, then saving it to csv.

How to handle multiple fields of data that must be of certain type? Formatted in certain way - etc.

I have a hard time abstracting class structure for this problem.

What design pattern would suit me best?

Answer 1

I would not go as far as to think in design patterns initially, but rather decomposing your problem into smaller parts, at least at first.

Now, your description may be a bit vague to actually get into much detail about the actual processing, but from it I can identify at least three parts:

• A data reader
• A data processor
• A data writer

You may want to think of both your readers and writers similar to how you would handle files, or streams. For example, with your reader could have:

• Open
• Read (read just one item)
• Close

And the same for the writer, but with Write.

Abstract two interfaces, an IDataReader, and an IDataWriter, and have your IDataProcessor interact with both the reader and the writer, but only through interfaces.

Having those contracts in place mean you can have one implementation for a reader, such as XLSReader : IDataReader, and CSVWriter : IDataWriter for the writer, without the processor having to worry about their internal ways to solve each one.

Having these abstractions let you write clean and independent unit tests for each part, in isolation (by mocking the other parts), which will let you test every step of the way during development.

Also, these type of contracts let you process data in small packets, instead of one big bulk of data (like a large DataTable and things like that), measure progress, etc.

Now, depending on your actual requirements, you can have schemas or definitions your reader provides of the data types, the same for outputs, formatters and transformers that could alter incoming data and build the output, etc.