Could anyone help me with the following problem statement?
I have a folder where an external system deposits some flat files which needs to be “picked” up “processed” and “persisted” by the system we want to build
The files are simple ASCII files with lines & columns of data (column separator being used is a comma)
The files could be divided into multiple categories. The processing of the data within the file would depend on the category it falls under (processing logic is not important as long as the design of the processing unit is fleshed out)
Finally the processed data needs to be persisted into the database (the data design need not get into field level details for the processed data storage, as long as the table level ER is delved into)
Please guide in designing & tips to solve it. or any book reference.