The perfect match: model and algortihm

Today is a Friday, so a good day to blog about this kind of topic. It's something I have been wondering about for a while and occasionally made some effort to make progress on or find an answer to, but never with much success.

Here is the problem, and one I have come across in numerous contexts: I have a model, let's call it $M:R\rightarrow S$ and an algorithm, let's call it $G:R\rightarrow R$. I want to optimise the values of M such that the model fits some data. Different ways to express this would be, for example, as minimising squared difference or maximising a likelihood function.

This problem can be solved in various ways. The principal three I've seen used are some kind of steepest descent (what I call deterministic), a statistical method such as Markov chain Monte Carlo or simulated annealing (statistical) and genetic algorithms.

My question is whether, given a model, it is possible to predict which algorithm will solve the problem the best. For each of the three types of algorithm above, I have heard exponents claiming that theirs is the best, to the point that you would think it foolish to even attempt to use any other algorithm. I can easily think of problems that are particularly suited to each one, but the question is where does the boundary lie between the effectiveness of one algorithm and another? If I invent an entirely new algorithm, what properties do I need to check to see if it works well?

The answer is almost certainly that it is very difficult to tell, but can you be sure? If so, how sure can you be, and is there any good reason not to use algorithm G on problem M?

So, my applied mathematician friends, has this problem already been solved? I think my issues with trying to found out lie in the fact that algorithms come from different fields using different terminologies. What language would you use to express this problem? What sort of tools would you use to solve the problem? If no such method could exist, can you prove it?

Here is what I envisage an answer might look like: the algorithm I use has a set of scores associated with various properties, such as calculations per step, difficulty experienced by a local minimum, speed at solving a standard problem. Given my model, I can sample the domain of parameters a finite number of times to get certain properties (or even analytically find a value) e.g. does it have local minima, how smooth is the resulting space, are there large local gradients. By multiplying or combining in some way corresponding properties of algorithm and model you assess the suitability of the algorithm to the model (e.g. the algorithm scores 5 for problems with local minima, model has a score of 3 for local minima, so add 15 to final score).

Anyone interested in trying to find a solution?