Are there any good methods for doing this? I can find papers with algorithms, but they usually do not show their computational results with the same problem sets and the publishing year of two papers is usually different so they will most likely use different equipment for solving the problems.
Edit: By problem, I am referring also to the specific size of the problem, memory usage and other characteristics of the problem.
Regarding the difference in setting, generally the "well-acclaimed" SOTA papers don't have these issues. At least that's what I've noticed majorly in my field.
If that is the case, then I suggest making a small study within the paper covering all the results in the different settings. (Please do correct me if I'm wrong here).
I think you have a misunderstanding about algorithm efficiency and its study. A good course in algorithms and data structures should correct that.
I'll assume here that you actually mean algorithm and not just a useful computer program. They are not the same thing. Some useful programs, while they may contain algorithms, have their performance dominated by other non-algorithmic concerns. The performance there might, indeed, be tied to a particular kind of dataset for which small deviations result in wildly different performance.
Algorithms are judged good or bad based on theoretical concerns, not by running them on sample data. The theoretical measures make them independent of hardware within certain bounds. A fast algorithm will run faster than a slow algorithm on different computers that have a similar architecture (see caveats below). Some algorithms (bubble sort) are always bad. Others are always better (selection sort). Still others are "always" good (merge sort, but see caveats).
So, the "different equipment" idea is a red herring, provided that the equipment has a similar architecture.
Caveats, and there are quite a few.
Some algorithms require a lot of memory (quick sort on a large set) because they are highly recursive. If a machine has insufficient fast memory then quick sort on a large data set will be limited by the, independent, efficiency of the memory system.
Some "slowish" algorithms work well on small data sets and can be preferred over "fastish" algorithms if the data set is expected to be small. But these things are known and are normally incorporated into working software. For example, in quick sort once a "split" results in a segment of size 10 or so, a quadratic algorithm will take over to complete the sort. But that is just good software design, and it uses known, theoretical, results for the various options.
However, if the architecture of a machine changes drastically (as a GPU differs from a CPU), then the game changes and some algorithms that are fast on one will be slower on the other. For example, still sorting things, if an architecture makes comparisons "free" then an algorithm that uses lots of comparisons, rather than rearrangements, will be preferred on that architecture. But, again, this is based on the theoretical considerations that measures the various kinds of operations required by an algorithm.
So, to answer your question, you need to look at the theoretical measures, and these are normally provided by authors.
Another caveat. There are some applied problems that are fairly ill structured and haven't (as yet) yielded to the theoretical analysis assumed above. Perhaps a bit of chaos is involved. For these, some empirical measures might be required at the moment. But some of those sorts of problems can also drive people to invent new architectures that can efficiently handle the problems. But, the same sort of problem also drives the theoreticians to invent new algorithms that are amenable to a general analysis of efficiency. But, the results will, most likely, be reported in theoretical terms, not timings on specific hardware. This is probably what you are seeing. And, hopefully, part of their analysis includes limits on when the algorithm can be expected to behave well.
