Developing and applying an assessment framework for climate services quality and maturity

Climate services involve the generation, provision, and contextualization of information and knowledge derived from climate research for decision making at all levels of society. These services are targeted at informing adaptation to climate variability and change, widely recognized as an important challenge for sustainable development. While there is a wealth of climate services available, ranging from web sites serving data through to bespoke, industry specific applications to support decision making, there is generally little information that objectively characterises the quality and maturity of a climate service. This poses a challenge for both climate service users who wish to understand the quality of the service being provided, and for climate service providers who wish to be able to differentiate their service from those with lower levels of maturity and quality. This research will empirically investigate the quality and maturity of several climate services, including from the UK Met Office, through interdisciplinary climate science research.

More details:

Stage 1 (6 months): Understanding of Quality and Maturity in Services and Technology. This stage of the work will investigate how quality has been assessed in different service sectors, e.g., health care, education and finance. Different quality dimensions – e.g., effectiveness, legitimacy, fitness for purpose, value for money, perceived quality, performance and reliability – will be examined. The characteristics and potential indicators of different quality dimensions will be studied across different service sectors. Understanding the maturity of technology will involve the investigation of (a) the concept of Technology Readiness Levels (TRLs) for the space industry; (b) their adoption across a wider range of sectors; and (c) the practicalities, benefits, and issues with their use.

Expected outcome: a baseline understanding of how assessments of service quality and TRLs have been developed and applied in practice, and an understanding of the challenges in applying a similar concept to a new area of service delivery.

Stage 2 (6 months): Developing a framework for the assessment of climate service quality and maturity. The first step in developing a coherent framework is to understand the needs for such information arising from both the user and service developer communities. Based on understanding the development and application of service quality and TRLs assessments, this task will involve the development of an initial assessment framework to take to these communities to explore their potential use and benefits. This consultation is expected to lead to an iterative development of the indicators of the assessment framework to better meet the user and service developer requirements.

Expected outcome: a preliminary assessment framework that can be applied in the third stage of the work.

Stage 3 (15 months): Applying the framework to climate service projects. This stage of the work will take the preliminary assessment framework developed in Stage 2 and will apply it initially to a small selection of Met Office climate service offerings. Particular examples may include the application to two contrasting services provided to the transport sector: seasonal forecasts of winter conditions provided to transport planners; and assessment of the climate resilience of transport networks. The former represents an experimental service under development, the latter is a well-established and long-standing service. The application of the framework to services with contrasting characteristics will test its ability to distinguish between services with differing levels of quality and maturity. The resulting assessments will be tested with the users and service developers for the selected example services. Again, it is anticipated that this application of the assessment framework will lead to further refinements to the indicators.

Expected outcome: The outcomes are expected to be two-fold: firstly, a further refined assessment framework based on insight arising from the practical experience of applying the preliminary framework to real projects with different characteristics; secondly, an evaluation of the utility of the resulting assessments that are produced for the projects.

Stage 4 (9 months): Analysis and write-up. This stage will analyse and reflect on the practical experience of applying the preliminary assessment framework to real projects and the feedback received from the users to propose a final assessment framework that contributes to both climate service theory and practice. Thesis chapters will be finalised during this stage.