Q&A: how much evidence is sufficient for decision-making?

I was recently asked this question:

Do you have a rule of thumb for deciding how much evidence is sufficient for decision making? I often see decisions made on the basis of a sample of one.

Here’s my answer: