Quality Indicators

A research registry is only as good as the information it contains.  Readers need complete and clear information with the central principle being transparency.  We are dependent on the integrity of the researchers for input and cannot independently verify what they have entered.  How can we judge quality and what should the quality indicators (QI) for the Research Registry be?

We cannot come to conclusion on this until we discuss how a research registry entry should be strutured.  So what should researchers look to publish in a registry and how should it be structured?  In the 1940s, Austin Bradford-Hill pioneered the IMRAD structure – introduction, methods, results and discussion – preceded by an abstract. This structure reflects the process of scientific discovery.  This format is now a worldwide convention and enshrined in guides for authors and uniform requirements from the ICMJE.[1] His explanation for this rationale was provided in a speech to the World Medical Association, where he argued that each section corresponds to the answers to four questions a reader needs to know[2]:

  1. Why did you start?
  2. What did you do?
  3. What did you find?
  4. What does it mean?

The elegance and simplicity of the structure has meant it has lasted the test of time.  So what does this mean for research registration? Research registries do not ask whether a particular piece of research should have been done, people have the right to register ‘bad’ research. Their research may go on, never to be published and universally panned by peer-reviewers – but they must be allowed to register it.  Posting results is not a legal requirement for authors registering research but is recommended once they are available and discussion or what it means is more appropriate for a journal publication – which registries complement and don’t replace.  The key questions for us are therefore:

  1. Who did the research?
  2. Where did they do the research?
  3. Why did they do it?
  4. What did they do? This can be expanded to include the PICO items.
  5. When did they do it?

These four questions can be mapped to our registration form.  Hence quality relates to the completeness of the answers to these questions.

The completeness and clarity of these fields in particular will give us an assessment of the quality of an individual entry.  Their completeness over a series of consecutive records will give us a quality indicator over a period of time.  This is something that can be tracked.  Indeed a composite score out of nine (for the nine unique elements on the right hand side of the table) can be calculated as the QI and compared over time to see how the quality of the registry changes over time.

 

References
[1]. ICMJE. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly work in Medical Journals [online]. Available at: http://www.icmje.org/recommendations (accessed 19 July 2015).

[2]. Hill AB. The reasons for writing. British Medical Journal 1965;iv:870.