Loading...

Get started

DMAT is organised around eight ‘domains’ that contribute to good decisions about health funding. Each domain contains a number of questions for you to answer as an individual or as a group. When you have finished, your answers are brought together in a visual report. To complete the tool and generate a report, you must provide a response to the questions in all eight domains. If you feel unable to answer a particular question, you can click the ‘Don’t know’ button. For further instructions, go to ‘How to use’.

How to use Privacy policy

Login

Are you commenting on this organisation for the first time?

YesNo

Yes

Are you answering as an individual or a group?

IndividualGroup

If you are commenting on a new organisation but this is not your first time using the tool, you can use the same email address as before to gain a new user ID.

Get started

If you are commenting on a new organisation but this is not your first time using the tool, you can use the same email address as before to gain a new user ID.

You may use any combination of upper and lower case letters in your group name and should include at least one letter. The group name needs to match exactly with others in your group in order to be able to compare results.

Tip: to ensure the name matches, get a member of your group to email the name around to everyone in your group.

Get started

No

If you’ve completed the tool previously, your unique user ID was emailed to you. Check your email records. The email comes from DMAT@kcl.ac.uk.

Go to your results

1. Institutional setting

Good decision-making about health priorities requires an understanding of an organisation’s role and obligations in the wider healthcare context. Public organisations such as clinical commissioning groups (CCGs) in England have a number of legal duties such as:

  • Being open or transparent about how money is spent.
  • Being explicit about how non-discrimination laws are complied with.
  • Consulting patients, the public and organisations which may be affected when big decisions are made.

You can read more about these legal duties here.

Ask the following three questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

1.1 Is information about the organisation's legal responsibilities and duties in commissioning (buying) healthcare services publicly available and easy to find?

Questions to help

For example, you could think about:

  • Where can people find this information?
  • Are there certain groups of people who might have difficulty finding this information?
  • Does the organisation make it clear what it is obliged to do (legal duties) and what it is doing voluntarily (in addition)?

1.2 Does the organisation demonstrate how it meets its legal responsibilities and duties when commissioning (buying) healthcare services?

Questions to help

For example, you could think about where and how the organisation provides regular, accessible reports on:

  • Financial performance.
  • Clinical standards and patient safety.
  • Action taken to promote diversity.
  • How it has met its duty to engage patients, the public and other stakeholders when making decisions that affect them.

1.3 Is the organisation clear about its relationships and collaborations with other organisations when making decisions about which health services it commissions?

Questions to help

For example, you could think about:

  • The different reasons there can be for working with other organisations, such as:
    • To improve efficiency or avoid duplication.
    • To improve standards of care.
    • To test or invest in different ways of providing services (for example, through community or voluntary sector organisations).
    • To meet national guidelines, such as requirements to collaborate on regional programmes.
  • Whether these reasons are fully understood by those involved in making the decisions.

2. Transparency

Being open and transparent is about showing a willingness to demonstrate and explain what has been done and why, and ensuring that people know when, where and how they can get involved or find out about decisions which are important to them.  It can help organisations assure themselves and others that the ways they make decisions are fair and unbiased.

Ask the following four questions to help make your decision or judge how well the  organisation is doing.

The questions are linked: read them all before answering each one.

2.1 Is information about the organisation publicly available and easy to find?

Questions to help

For example, you could think about:

  • Whether information about how the organisation is structured is published on its website or elsewhere, including the names and contact details of senior staff.
  • Where, how and when the organisation makes information about decision-making available, such as via community notice boards, or local media.
  • Where different groups usually go to find information – and is it accessible to them?

2.2 Is information about the organisation’s structure, its decision-making criteria and important dates and events understandable?

Questions to help

This question is about situations in which information might be available, but may not be understandable for a large number of people affected.  For example, you could ask yourself:

  • Does the language used to explain decisions include too much expert jargon?
  • Are the documents provided too long for the intended audience?
  • Does the organisation provide an explanation of difficult words, glossary of terms or a summary of long meetings and documents?

 

2.3 Does the organisation provide reasons for its decisions about which health services to fund?

Questions to help

This question is about whether the organisation explains clearly – and publicly – the individual reasons (or criteria) behind their decision-making, and also how the reasons are used and balanced against each other.

For example, you could think about:

  • Does the organisation use the following as a basis for decisions:
    • Cost effectiveness.
    • Budget impact.
    • Whether the decision affects the local population fairly (equity considerations).
    • Quality of care.
    • Other reasons that have been agreed.
  • Does the organisation explain clearly why and how it has weighed the importance of one reason against another (for example, a decision to fund a service that has high costs but helps a previously disadvantaged group).

2.4 Does the organisation offer legitimate reasons for its decisions?

Questions to help

This question is about how confident you are that the organisation offers legitimate reasons for its decisions: that is, reasons that can be explained and justified. There is no single all-encompassing definition of legitimacy. However, you could ask:

  • Do the reasons given reflect the remit and legal duties of the organisation?
  • Are the processes consistent across different funding decisions?
  • Have any important criteria been overlooked?
  • How are criteria balanced against each other – is it clear?

3. Accountability

Accountability requires an ability to give reasons for and justify one’s decisions. Accountability can take different forms, and healthcare commissioners are accountable to a great number of people and organisations and in different ways, locally and nationally. Sometimes accountability is formal – legal or financial accountability, for example. Sometimes it is less formal – accountability to colleagues, for example.

This domain is closely related to Domain 2 – Transparency, as one way of being accountable is to be open about the basis for decisions.  Ask the following two questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

3.1 Does the organisation state clearly to whom it is accountable?

Questions to help

This question is about how the organisation understands and approaches accountability.  For example, you could ask:

  • Has the organisation identified to whom it is accountable, for example in its strategy documents?
  • Is the organisation open about whom it is accountable to?
  • Is accountability discussed as a stand-alone subject in the organisation’s reports or documents?

3.2 Does the organisation demonstrate that it fulfils its duties with regard to accountability?

Questions to help

This question is about how the organisation shows that it fulfils its duties with regard to accountability. For example, you could ask:

  • Is the organisation transparent about its decisions (see also Domain 2 Transparency)?
  • Does the organisation use diverse communication opportunities to reach a range of audiences, such as regular stakeholder forums, public meetings, newsletters (see also 4.3 Participation and consultation)?

4. Participation and consultation

In addition to providing accountability (see Domain 3 – Accountability), the views of patients, health professionals, community organisations, elected representatives and the public are important because they provide additional and diverse perspectives on decisions about which health services to fund.  Enabling different groups to contribute also helps ensure that distinct and special needs are understood. In England, health commissioners have a statutory duty to consult stakeholders when making certain funding decisions.

 Ask the following five questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

4.1 Does the organisation consult all necessary groups (as is legally required)?

Questions to help

This question refers to the statutory duty on health commissioners to consult a wide range of stakeholders who may be affected in different ways by decisions about funding.  To help answer this question, you could ask:

  • Is the organisation clear and open about any statutory obligations it has to consult about funding decisions?

4.2 Is information on the ways in which patients, members of the public, health professionals and others can get involved publicly available and explained?

Questions to help

For example, you could think about how easy it is for people to find out how to get involved (if they want to), such as via:

  • The commissioner’s and health providers’ websites.
  • A named contact person and phone number.
  • Public library or community notice boards.
  • Newsletters.
  • Articles or advertisements in local media.

4.3 Does the organisation use a wide range of methods to consult and engage with stakeholders and the public?

Questions to help

There is no single right or wrong way to consult or involve different groups, and different approaches have advantages and disadvantages depending on what you are trying to achieve.  For example, you could think about whether the organisation offers a range of methods to hear people’s views on decision proposals, such as:

  • Surveys.
  • Public consultations and hearings.
  • Regular stakeholder forums.
  • Written consultations.
  • Public meetings with question and answer sessions.
  • Deliberative forums.

 

4.4 Is the organisation transparent about how the views of patients, the public, health professionals and other stakeholders influence the ultimate decisions?

Questions to help

This question is about whether the organisation communicates the ways in which it has taken different views into account (alongside any other factors that affect the final decision).  For example, you could think about:

  • How the organisation feeds back to people who contributed views regarding how their views were incorporated.
  • Whether the organisation feeds back how views were not incorporated and why.

4.5 Does the organisation have a good strategy for ensuring a wide range of views can be heard from patients, members of the public, health professionals and other stakeholders?

Questions to help

This question is about the organisation’s overall goals and methods for hearing and incorporating people’s views when making decisions. For example, you could think about whether the organisation:

  • Ensures that the voices of minority groups are heard.
  • Allows enough time for participants to reply to consultation questions.
  • Provides sufficient and understandable documentation to help people get involved with decision-making.

5. Clinical effectiveness

Clinical effectiveness means a treatment, treatment ‘pathway’ or service works well and does what it intends to do. Resources should not be allocated to care that does no good, or worse, that does harm. However, knowing what is effective is not easy. Interpreting research results can be hard and there are still well-established forms of care for which there is no or limited evidence, for example because they have never been tested in clinical trials.

Public health and social care services are often based on different kinds of evidence and not clinical trials. However, the principle that resources should only be allocated to care which is known to be effective still applies and we recommend that you think about these services in a similar way.

Ask the following three questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

5.1 Does the organisation have a system in place to identify relevant national guidance or standards?

Questions to help

Many countries have established organisations that develop national guidelines on the effectiveness of health care services and drugs. In England, the National Institute for Health and Care Excellence (NICE) makes recommendations on services and treatments based on an assessment of available scientific evidence.

For example, you could ask:

  • Is there someone whose job it is to help keep the organisation up to date about the latest national guidance?
  • Does the organisation have a database or register of new and recent guidance?

5.2 Does the organisation have a system in place to manage uncertainties about, or unavailability of, evidence on clinical effectiveness?

Questions to help

This question concerns the quality and relevance of some information about clinical effectiveness – and the difficulties there can be when it comes to making decisions regarding such.  This is often the case for services that have been provided for decades or more, particularly where the decision to fund them was based on criteria that is different to that which would be applied now.  For example, it may be that there are no studies on a service’s or drug’s effectiveness, or that the quality of the studies is not clear.  Sometimes, it may not be easy to assess how the evidence is relevant to your local context.

It is important that organisations are clear and open about how a decision was reached in these circumstances. For example, you could ask:

  • Did the clinicians and other health professionals involved in the decision reach agreement on the quality of evidence?
  • Was other evidence considered, such as patient experience?
  • Were current or potential service providers invited to provide information about local outcomes (health benefit or improvement)?

5.3 Does the organisation have a system in place to stop funding clinically ineffective services or treatments?

Questions to help

If clinical effectiveness is a criterion or reason for commissioning a service, then it should follow that steps will be taken to stop funding a service or require changes to be made if it is clinically ineffective (sometimes called ‘decommissioning’).  In practice, this is often not done for a variety of reasons – such as limited capacity to undertake the work, or the challenge of restricting services which patients have been using for a long time.

For example, you could ask:

  • Does the organisation have a clear and published policy not to fund services or treatments that are clinically ineffective?
  • Does the organisation have a strategy or position statement regarding decisions to stop funding existing services or treatments that are not clinically effective?
  • Does the organisation require providers to undertake service reviews focusing on clinical effectiveness, to support any case for future funding?

6. Cost effectiveness

Cost effectiveness examines the costs of a service or treatment in relation to its benefits in order to assess whether the costs of funding a service can be justified in light of the expected benefits. Cost effective judgements centred on ‘value for money’ can be controversial. For some, it means that there is a risk that financial considerations could be put before patients’ needs. For others, it means that the needs of all patients, rather than a few, are considered and the best possible care for the largest number of patients is secured. Even when sound health economics methods are used to assess cost effectiveness, a decision has to be made on how much benefit is ‘enough’ benefit to justify costs.

Ask the following three questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

6.1 Does the organisation have a system in place to collect and evaluate evidence in order to ensure that what is commissioned is cost effective?

Questions to help

Similar to evidence for clinical effectiveness, evidence for cost effectiveness comes in different forms and from different sources.  Given that most health systems and commissioners operate under strained budgets, how these organisations operate with regard to cost effectiveness is an important marker of the organisation’s decision-making.

Depending on the type of service or treatment in question, you could ask:

  • Does the organisation make use of guidance on the cost effectiveness of pharmaceutical treatments from NICE?
  • Does the organisation make use of evidence or guidance from other national organisations such as the Social Care Institute for Excellence (Scie) or Public Health England (PHE)?
  • Does the organisation look at academic literature regarding the cost effectiveness of treatments where national guidance does not exist?

6.2 Does the organisation have a system in place to manage uncertainties about, or unavailability of, evidence on cost effectiveness?

Questions to help

As with question 5.2, this question concerns the quality and relevance of some information about cost effectiveness – and the difficulties there can be when it comes to making decisions regarding such.  This is often the case for services that have been provided for decades or more, particularly where the decision to fund them was based on criteria that is different to that which would be applied now.  For example, it may be that there are no studies on a service’s or drug’s effectiveness, or that the quality of the studies is not clear.  Sometimes, it may not be easy to assess how the evidence is relevant to your local context.

It is important that organisations are clear and open about how a decision was reached in these circumstances. For example, you could ask:

  • Did those involved in the decision reach agreement on the quality of evidence?
  • Was other evidence considered, such as patient experience?
  • Were current or potential service providers invited to provide information about the costs of alternative treatments and any difference in benefits gained?

6.3 Does the organisation explain how it considers the financial implications of each decision?

Questions to help

Cost effectiveness looks at whether the benefits of a particular service justify its costs.  At the same time, a decision to fund one service may mean that there is less money available to fund another or existing services, especially for organisations that have a fixed annual budget .

For example, you could think about:

  • Does the organisation consider what it would mean for other services when deciding whether to fund a new service or treatment?
  • Is the organisation clear and explicit about the impact of a decision to fund a service or treatment on the total budget?

7. Quality of care

There are different definitions of quality of healthcare, but patient safety, clinical effectiveness and patient experience are usually at the heart.  A health care service is thought to be of high quality when the highest standards are met in each of these areas.  However, it is important to recognise that individual patients, professionals or commissioners may have differing opinions on the standards that a service needs to meet in order to be considered a high quality service. For example, some argue that timely access to, or availability of, services when patients need them should also be considered a measure of quality. Regardless of different understandings of the term, it almost always includes a focus on the experience and the safety of care for individual patients.

Ask the following three questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

7.1 Is information on the organisation's understanding of what makes good quality of care publicly available and easy to find?

Questions to help

For example, you could ask:

  • Does the organisation include a definition of quality of care in key documents about strategy and values?
  • Does the organisation have a strategy or plans to review quality of care provided by the services it funds on a regular basis?

7.2 Is information on the quality performance of the services that the organisation commissions publicly available and easy to find?

Questions to help

For example, you could ask:

  • Does the organisation have a Quality of Care policy?
  • Do contracts with service providers include Quality of Care requirements (based on a policy, if there is one)?
  • Does the organisation have a Quality of Care committee which publishes reports?

7.3 Does the organisation have systems in place to identify and follow national quality of care initiatives?

Questions to help

For example, you could ask:

  • Is there someone whose job it is to help keep the organisation up to date about the latest initiatives?
  • Does the organisation have a database or register of new and recent guidance or good practice regarding quality of care?

 

8. Fairness

Fairness means different things to different people. Some people talk about ‘equity’, others about ‘justice’ or ‘rights’. In healthcare prioritisation, fairness can mean:

  • Treating everyone with equal concern and respect.
  • Positive action to reduce inequalities, for example by providing services for a previously overlooked group.
  • Fulfilling the legal duty not to discriminate against people on the basis of age, gender, ethnicity, religious background, sexual orientation or other characteristics.

The challenge of how to be fair to all is at the heart of any healthcare organisation’s decision-making about what services or treatments to prioritise when resources are constrained.

Ask the following two questions to help make your decision or judge how well the organisation is doing.

The questions are linked: read them all before answering each one.

8.1 Does the organisation demonstrate that it has policies in place to identify equality and diversity concerns that may arise from its decisions and strategies?

Questions to help

For example, you could ask:

  • Does the organisation routinely carry out Equality Impact Assessments with regard to significant investment or disinvestment decisions?
  • Does the organisation have a strategy to address health inequalities in the local area?

8.2 When services are prioritised for special patient or population groups (children or older people, for example), does the organisation explain the reason for this?

Questions to help

The organisation could have a number of different reasons, depending on local circumstances. For example:

  • The disease or condition that a service will treat is only – or mainly – experienced by a specific group.
  • There is a longstanding lack of services in the area for the group in question.
  • Services currently provided are in unsuitable or inaccessible locations for the group.
  • A need to provide services for the group has been identified by the local Joint Strategic Needs Assessment (JSNA), or a similar exercise.

1. Institutional setting

Av scoreAvg

1.1 Is information about the organisation's legal responsibilities and duties in commissioning (buying) healthcare services publicly available and easy to find?

1.2 Does the organisation demonstrate how it meets its legal responsibilities and duties when commissioning (buying) healthcare services?

1.3 Is the organisation clear about its relationships and collaborations with other organisations when making decisions about which health services it commissions?

2. Transparency

Av scoreAvg

2.1 Is information about the organisation publicly available and easy to find?

2.2 Is information about the organisation’s structure, its decision-making criteria and important dates and events understandable?

2.3 Does the organisation provide reasons for its decisions about which health services to fund?

2.4 Does the organisation offer legitimate reasons for its decisions?

3. Accountability

Av scoreAvg

3.1 Does the organisation state clearly to whom it is accountable?

3.2 Does the organisation demonstrate that it fulfils its duties with regard to accountability?

4. Participation and consultation

Av scoreAvg

4.1 Does the organisation consult all necessary groups (as is legally required)?

4.2 Is information on the ways in which patients, members of the public, health professionals and others can get involved publicly available and explained?

4.3 Does the organisation use a wide range of methods to consult and engage with stakeholders and the public?

4.4 Is the organisation transparent about how the views of patients, the public, health professionals and other stakeholders influence the ultimate decisions?

4.5 Does the organisation have a good strategy for ensuring a wide range of views can be heard from patients, members of the public, health professionals and other stakeholders?

5. Clinical effectiveness

Av scoreAvg

5.1 Does the organisation have a system in place to identify relevant national guidance or standards?

5.2 Does the organisation have a system in place to manage uncertainties about, or unavailability of, evidence on clinical effectiveness?

5.3 Does the organisation have a system in place to stop funding clinically ineffective services or treatments?

6. Cost effectiveness

Av scoreAvg

6.1 Does the organisation have a system in place to collect and evaluate evidence in order to ensure that what is commissioned is cost effective?

6.2 Does the organisation have a system in place to manage uncertainties about, or unavailability of, evidence on cost effectiveness?

6.3 Does the organisation explain how it considers the financial implications of each decision?

7. Quality of care

Av scoreAvg

7.1 Is information on the organisation's understanding of what makes good quality of care publicly available and easy to find?

7.2 Is information on the quality performance of the services that the organisation commissions publicly available and easy to find?

7.3 Does the organisation have systems in place to identify and follow national quality of care initiatives?

8. Fairness

Av scoreAvg

8.1 Does the organisation demonstrate that it has policies in place to identify equality and diversity concerns that may arise from its decisions and strategies?

8.2 When services are prioritised for special patient or population groups (children or older people, for example), does the organisation explain the reason for this?

Any suggestions?

Go to feedback