Conflicting Evidence in Substance Studies

One of things that struck me as I looked into the use of caffeine as a sports supplement was the lack of a study that you would consider as definitive as well the conflicting nature of many reports. In particular caffeine's efficacy in interval type sports was varied as well as the debate around how caffeine and creatine interact with each other.

Considering caffeine is almost certainly going to be one of the most studied and popular supplements I look at, I imagine it will be hard to draw accurate conclusions on the definitive nature of their effects.

This doesn't dissuade me from the quest to find a sports nootropic, it just reinforces that I will have to be vigilant in my reviews, particularly in some of the more out-there supplements and herbs.

Here's a few things I plan to keep in mind while reading the research:

• Publicly available. That way you can check it out too and hopefully I can get some insight from any interested readers.
• I want it to be designed well i.e. double-blind, placebo research. Research where the subjects or experimenters can sway the result isn't likely to impress me or provide the greatest results.
• I want decent sample sizes. Testing only 10 or so people wont exactly represent an substance's effect on the wider population.

Edit: I came across this a bit in my current research reviews so I'm adding it in now:

• Human trials. The value of testing a cell in a labratory or even just in rats is of little application until further, specific research is carried out.

Sources and further reading:
http://www.ncbi.nlm.nih.gov/pubmed/19088794