GRF · Research Nexus
Research methods dialogue
Risk evidence comes from experiments, observational studies, models, case studies, expert judgement and lived experience, and each answers some questions well and others poorly. Methods dialogue is the comparison of those designs: what each assumes, where each fails and how evidence from different designs can be weighed together. This page sets out why methods choices carry governance consequences, how their quality can be checked, and who decides which evidence counts.
Research methods dialogue on the record.
Research methods dialogue: how the risk develops.
Comparative methods and limitations. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Hierarchies are misapplied
Evidence hierarchies built for clinical questions are applied to policy problems where trials are impossible or unethical. Relevant evidence from other designs is discounted, and decisions wait for evidence that cannot exist.
Assumptions stay hidden
Every model and estimate rests on assumptions about data, mechanisms and context. When these go unstated, users cannot judge transfer, and disputes about results are really unacknowledged disputes about assumptions.
Flexibility inflates findings
Many analytic choices remain open after data are seen. Choosing among them by result produces findings that do not replicate, and the problem compounds across a literature that favours positive results.
Context does not travel
An effect estimated in one setting depends on its institutions, population and timing. Transferring it without stating those conditions misleads decisions elsewhere, especially when evidence moves between very different settings.
Research network directory.
The screen a team would use to follow this subject: its measures, states and decisions. Names are examples and values are illustrative.
Research network directory
Research methods dialogue · Research network A · Year 2
Collaboration graph
nodes: institutions · illustrative
Joint projects
| Compound risk methods | 4 | Active |
| Local knowledge in warnings | 5 | Proposal |
| Open data for risk | 10 | Reporting |
What is measured, and how well.
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Studies with preregistered protocols | percent of studies on a theme | annual | Registry and publication records | OBSERVED |
| Replication and reproduction attempts | count per theme | periodic | Replication databases and journals | REPORTED |
| Outputs sharing analysis code | percent of outputs | annual | Publication and repository metadata | OBSERVED |
| Syntheses rated with a stated certainty framework | percent of syntheses | annual | Synthesis libraries | OBSERVED |
| Mix of study designs on a theme | count by design type | periodic | Evidence and gap maps | ESTIMATED |
| Studies reporting sensitivity analysis | percent of studies reviewed | periodic | Content review | ESTIMATED |
How research methods dialogue is governed: instruments, mandates and commitments.
The instruments, mandates, actors and accountability arrangements a policymaker, diplomat or development agency works with, citing the agreements that set them. Structures follow the frameworks named. Any state or progress drawn is illustrative, not a record.
Who holds which role
The actors around research methods dialogue: who decides, who implements, who finances and who brings knowledge and voice.
Where the evidence comes from.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Bron | What it gives | Licence class |
|---|---|---|
| Campbell Collaboration evidence and gap maps | Study designs and coverage across social policy questions | Open access, attribution |
| Cochrane Handbook and methods guidance | Standard methods for systematic reviews and their known limitations | Free to read; reuse terms per item |
| GRADE working group guidance | A framework for rating the certainty of evidence and the strength of recommendations | Free to read, working group terms |
| OSF Registries | Preregistered protocols and registered reports | Open, terms of service |
| FORRT Replication Database | Records of replication attempts in the social and behavioural sciences | Open, terms per release |
| EQUATOR Network reporting guidelines | Reporting standards by study design, such as CONSORT and STROBE | Free to read, terms per guideline |
| OECD Global Science Forum reports | Policy analysis on reproducibility and research infrastructure | Free to read, OECD terms |
Who decides, and what the evidence serves.
Research Nexus prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Wie besluit | The question | Evidence this page serves |
|---|---|---|
| Research funders | Whether to require preregistration, data and code sharing, or to fund replication | Comparative policies; evaluations of their effects with stated limits |
| Evidence synthesis and guideline bodies | Which study designs and certainty frameworks to use for a policy question | Methods comparisons; known failures of design hierarchies in policy settings |
| Government evaluation units and statistical offices | Which evaluation designs to require for public programmes | Methods dialogue records; feasibility and ethics of experimental and quasi-experimental designs |
| Journal editors | Whether to adopt registered reports and reporting guidelines | EQUATOR guidance; replication evidence by field |
Outputs this topic produces
- Methods seminars comparing designs on a shared risk question
- Peer learning briefs on weighing mixed evidence for decisions
- Dialogue records of methodological disagreements and their sources
- Plain-language notes on the assumptions behind common methods
What methods dialogue asks.
- What kind of claim can this method support: correlation, causation, mechanism, possibility?
- How do results change when a different but defensible method is used?
- How can quantitative and qualitative evidence be combined without losing the strengths of either?
- What are the known failure modes of the methods most used in this field?
Why methods comparison matters.
Analyses of the same data by different expert teams often reach different conclusions, because each team makes defensible choices about variables, models and assumptions. This variation is a real source of uncertainty that single studies hide. Comparing methods openly reveals it and helps users judge how robust a finding is.
Different methods also answer different questions. A randomised evaluation can show whether an intervention worked on average in one setting; a case study can show how and why it worked; a simulation can explore what might happen elsewhere. Confusion arises when one kind of evidence is asked to answer another kind of question.
Why now: disagreements about risk often turn on method: models, definitions and uncertainty treatments that differ between disciplines. Research Nexus convenes method dialogues that make those differences explicit, records them with their consequences for decisions, and can propose shared definitions to the Nexus Standards.
Practices for comparing methods.
Methods comparison is most useful when it is routine rather than reserved for disputes. Several practices make it so, each with its own cost.
- 01
Methods seminars
Sessions where researchers present and challenge each other's methods. Limits: depend on openness and good faith.
- 02
Multi-team analyses
Several teams analyse the same question independently. Limits: resource-intensive.
- 03
Causal inference frameworks
Explicit causal assumptions and identification strategies, from causal inference and impact evaluation. Limits: assumptions cannot always be tested.
- 04
Mixed-methods synthesis
Combining statistical and qualitative evidence, with probability and statistics. Limits: no single agreed approach.
How it is studied, and what the methods cannot say.
- 01Causal inference and impact evaluationEvaluating funder policy changes · Assessing knowledge brokerage schemesApplied here
- 02Waarskynlikheidsleer en statistiekBuilding bibliometric indicators · Detecting selective reportingApplied here
- 03Policy, institutional and decision scienceComparing science advice systems · Mapping integrity proceduresApplied here
Questions and answers.
Is a randomised trial always the strongest evidence?
What is preregistration?
Why do methods disagree?
Is one method best for studying risk?
Why do expert teams reach different conclusions from the same data?
Can I present my methods at a seminar?
Are seminar records published?
Dra bewyse vanaf hierdie bladsy by.
Verslae is the Consortium's quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- E-posGateway GW-GRFStuur dit vanaf hierdie bladsyGetuienis, 'n regstelling, 'n geval, 'n metode, 'n beoordeling, 'n vertaling of profielinset. Dit kom aan gemerk met hierdie bladsy, sodat die redakteur wat dit besit dit ontvang.
- ZenodoOop deponeringDeponeer dit en dien dit in by die gemeenskapPubliseer u eie rekord op Zenodo met u ORCID iD, en dien dit dan daar in by die Reports-gemeenskap. Kurators beoordeel dit; die DOI bly joune.
- VerslaeGlobal Risks ForumSien elke manier om by te draDie bydraetipes, hoe elkeen beoordeel word, watter erkenning u ontvang, die lisensie en wat nooit aanvaar word nie.
Stuur nooit persoonlike data, sensitiewe infrastruktuurbesonderhede of enigiets wat u geen reg het om te deel nie. 'n Bydrae word voor gebruik beoordeel; die stuur daarvan verleen geen outeurskap of redaksionele regte nie, en borgskap koop nooit 'n bevinding nie.
Bring a governance question to the Forum.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal documents.
Show all 18 legal documents
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.