Some aspects of computer science research requires some form of implementation and an according evaluation. I have not yet seen where the minimum level of implementation expected for an academic paper is defined. For example, if a "novel" work requires a persistence layer and a researcher uses a file based method e.g. csv. Will that be wrong? Will his architecture point to a database or a file system ?
Your implementation needs to be good enough to evaluate the concept you want to evaluate - no more, but certainly no less. What this means in practice is largely dictated by what kind of evaluation or experiments you want to do. I have basically seen four broad classes in Computer Science:
- Proof-of-Concept (PoC): you only want to show that your idea can be implemented, so you only implement your idea (often in a fairly naive or bare-bones manner). Note that this is often good enough for a quick demo, but not good enough for a paper (because reviewers usually feel that the conclusion that something can be implemented is not satisfactory by itself).
- Evaluation Prototype: in addition to the PoC, you want to evaluate how well your idea works in some quantitative manner. For instance, if you are building some novel file system, you probably want to not only show that it can be implemented, but also that it works better in some dimension (e.g., faster) than existing systems. Your prototype can still be bare-bones, but you presumably need to build the core parts well because otherwise your evaluation may end up showing primarily that your prototype sucks. In a way, your prototype results show a sort of lower bound on how your idea works - that is, if your results are good your idea is validated, but if your results are not good, it may be because they idea does not work or your code is too bad of an implementation.
- User Study Prototype: in addition to quantitatively evaluating your prototype, you may want to run one or more case or user studies with it, or conduct a controlled experiment. Now you need to improve your evaluation prototype with at least a limited amount of bells and whistles, because it is hard to run an expressive user study if your system is almost unusable to anybody but yourself. In this stage you may be adding a GUI and a bunch of features that are unrelated to your science, but which may be needed even in trivial practical usage.
- (Simple) Product: the final stage is that you plan to actually release your implementation in some manner to the public. Maybe as part of an open source project, as a consulting delivery to a company, or even in the context of a spin-off company. You will now need to add many more bells and whistles unrelated to your science. Further, now you need to start caring about documentation, a release cycle, not doing breaking changes, etc., all of which are basically irrelevant as long as you are just building a prototype for your science.
So what does this mean for your project?
For example, if a "novel" work requires a persistence layer and a researcher uses a file based method e.g. csv. Will that be wrong? Will his architecture point to a database or a file system ?
I generally suggest my students to follow a lean approach: what kind of experiment do you plan to do? Will it matter what kind of persistence layer the prototype has? If no, then build whatever you can do quickest and where you are most confident that it will work correctly - but keep in mind that even the simplest prototype can sometimes run through all the phases above, so at least plan for the possibility that you may need to swap out your persistence layer for something a little more state-of-the-art than a flat text file later on.