Maturity of the science

The OECD Test Guideline (TG) or Guidance document (GD) development process is not the same as an academic project. To be agreed as a project within the OECD Test Guidelines Programme (TGP), a method should be demonstrated mature enough by a detailed explanation of its scientific validity for a particular regulatory purpose.

The first step towards developing a new or revised TG or GD is thus the definition of the test method (OECD GD1, revised in 2009). It needs to be clarified whether the test method is designed to predict, to measure or to represent. Additionally, a test method’s applicability domain must be defined. It refers to chemical classes, mechanisms of action and the range of responses for which the test is used reliably. The method proposed needs a sound scientific background (e.g. establish the mechanistic basis of the test and its relationship to the effect(s) of concern, and highlight the history of use of the test method, if any, within the scientific and regulatory communities). Awareness of relevant quality criteria and requirements for scientific data contributes to a solid demonstration of the maturity of the method prior to the OECD process. Furthermore, this helps TG/GD developers during the following project definition and development phases at OECD.

To establish the scientific validity of a test method, it is necessary to obtain information that fulfils the criteria and scientific principles for test method validation and to assess the extent to which the principles have been fulfilled. That is why even if a test method or test strategy does not need to be validated prior to being proposed for standardisation and harmonisation to the OECD TGP, its validation status, including how far it is mature, what is possible with the method and what further development is needed, must be described within the Standard Project Submission Form (SPSF), alongside the justification and rationale for the test method harmonisation (See Regulatory needs). 

“Solna Principles”

A set of principles for validation was developed at an OECD Workshop in Solna Sweden in 1996, to establish the scientific validity of a test method, however, the level of assurance that is appropriate for a specific purpose and type of test can vary and should be assessed on a case-by-case basis:
1) A rationale for the test method should be available. This should include a clear statement of scientific need and regulatory purpose.
2) The relationship of the endpoint(s) determined by the test method to the in vivo biological effect and to the toxicity of interest should be addressed. The limitations of a method should be described, e.g., metabolic capability.
3) A formal detailed protocol must be provided and should be readily available in the public domain. It should be sufficiently detailed to enable the user to adhere to it, and it should include data analysis and decision criteria. Test methods and results should be available preferably in an independent peer-reviewed publication. In addition, the result of the test should have been subjected to independent scientific review.
4) Intra-test variability, repeatability and reproducibility of the test method within and amongst laboratories should have been demonstrated. Data should be provided describing the level of inter- and intra-laboratory variability and how these vary with time.
5) The test method’s performance must have been demonstrated using a series of reference chemicals preferably coded to exclude bias.
6) The performance of test methods should have been evaluated in relation to existing relevant toxicity data as well as information from the relevant target species.
7) All data supporting the assessment of the validity of the test methods including the full data set collected in the validation study must be available for review.
8) Normally, these data should have been obtained in accordance with the OECD Principles of Good Laboratory Practice (GLP).

A retrospective assessment of the validation status of a test method can be carried out in the pre-OECD phase by reviewing all of the information and data available that supports the validity of a test method, including available peer-reviewed publications, data obtained in previous method development projects and already conducted (pre-)validation studies (e.g. intra-laboratory comparison). When conducting such an assessment, it may be necessary to address some questions. For example, can experimental data obtained with different variants of a test method protocol (e.g. use of different centrifugation speeds during a sedimentation step) be pooled together to assess the reliability of the method? Or Can a validity assessment be made using processed data when no raw data is available? 

A data gap analysis will allow TG/GD developers to identify additional data needs and prioritise the most important remaining gaps to fill during the development of the TG or GD, e.g. by an inter laboratory comparison. However, the OECD project needs to be built on a solid basis to be accepted by the Working Group of National Coordinators of the TGs programme (WNT). The maturity of the science at the beginning can be different for the different OECD WNT projects and can also determine the project outcome with respect to the development of a TG or GD. In case the test method or the scientific topic is not yet fully mature, a scoping review or review document can help to collect the state-of-the-art and bring together different stakeholders towards this topic. Once further mature, the test method could be implemented in a Guidance Document or if fully mature in a Test Guideline including an extensive validation study. See the NanoHarmony Training Material - From science to standards and harmonised OECD Test Guidelines to learn more about the different OECD documents.

• Not all science will be accepted for harmonisation and standardisation in the OECD Test Guidelines Programme. Publishing the results of pre-validation studies, initial method developments and data generated using the test method can support the integration of the test method in an OECD project. Additional proof for the maturity of the method should be sought.

• Statistical evaluations need to be appropriate. Statistical experts reviewing the project may request the use of particular tools and provide suggestions to improve the overall output of the results. 

• Pre-OECD phase: The maturity assessment of the scientific basis for a proposed test method should be assessed in the pre-OECD phase and improved during the project definition and development phases at the OECD.

• SPSF drafting: The maturity of science needs to be described in the SPSF. This includes providing data that has been generated within pre-validation studies or during the initial method development.