Partnership Approach, SCORE and Reports Webinar
- The Partnership Approach
- The SCORE Outcomes Framework
Partnership Approach, SCORE and Reports Information Session
Welcome to the information session on the Partnership Approach, SCORE and reports, during the session today we will discuss the following items. We are going to have an overview today of the Partnership Approach and the extended demographics. So we are going to look at how collecting these voluntary data items can benefit your organisation.
We will also discuss SCORE and take a look at three examples on how to record it. We will see how easy it is for your organisation to opt in to the Partnership Approach and start collecting this insightful and invaluable data as well, and we will have a look at the new reports for the Partnership Approach, so you will get a glimpse of what they look like at the moment.
Firstly, let’s discuss the value of measuring outcomes and extended demographics. The Partnership Approach is all about recording the client outcomes and this feeds into the bigger picture – which is about improving the lives of Australian individuals and families by ensuring their access to quality social services. The Partnership Approach is focused on outcomes measurement and the capture of client pathways data.
So this data will provide visibility of the outcomes achieved by clients and services so you will be able to see what they are achieving very clearly. You’ll be able to identify the combination of services that lead to the best results across the programs that you are delivering, and measuring outcomes also demonstrates the value of program investment in achieving your stated goals. Again from the reports you will be able to see clearly which reports (programs) are achieving the best outcomes to you clients and also to you, your organisation. It will also allow your organisation and the Department to have a better understanding of your client groups, demographic profile and complexity.
As many of you may are aware the Data Exchange is made up of four different parts. This diagram represents those parts and the framework of the Data Exchange. On the left hand side are the mandatory elements of the Data Exchange, they are made up of the priority requirements and Client Survey. So the priority requirements priority requirements are the basic data set that all service providers must report, and they’re captured under the concepts of clients, cases and sessions. Now this basic data is included in the standard report which is available to all users of the Data Exchange.
The Client Survey is currently in pilot phase. Now the Client Survey aims to collect client outcome and service experience feedback directly from clients. Once implemented, it will be mandatory for service providers to offer their clients the survey and ask if they would like to participate, so that’s why it sits in that mandatory part of the framework. So it is not going to be mandatory for the client to participate, so they can say either yes or no to their participation in the Client Survey.
On the right hand side in pink are the voluntary elements of the Data Exchange; the Partnership Approach. The Partnership Approach is focused on outcomes measurement and the capture of client pathways data.
It is entirely voluntary and can be entered into at any stage during the life of an organisation’s grant agreement. In exchange for opting in to the Partnership Approach and providing additional data, you will gain access to a series of Partnership Approach reports in addition to the standard reports. These can be used to assist with your business planning and support improvements in service delivery. The first of these reports are available now and these include the Resource Planning Report and Service Footprint Report.
More partnership reports are on their way, and will include information drawn from other government and population data sets such as ABS and SEIFA and Census information. Other reports will also include further client demographic information, service delivery and outcomes data.
So let’s take a look at the Partnership Approach in a bit more detail. Organisations that opt-in to the Partnership Approach enter the Priority Requirements data plus also a voluntary set of extended data comprising of what you can see on screen.
We do ask that those organisations that participate in the Partnership Approach that they agree to report SCORE outcomes for at least 50 per cent of their clients. Now you can see here on the screen the different domains of SCORE. They are:
- Circumstances – this will track a client's personal circumstances, such as housing, mental and physical health, or family functioning.
- Goals – can provide SCORE outcomes against a client's goals; whether they achieved or whether they’ve learned something due to contact with the service that you are delivering.
- Satisfaction – captures a client's satisfaction with the service they received, and;
- Community – captures outcomes of a group or community where it may be impractical to record outcomes for each individual participant.
Other data items as seen on the right hand side of the screen are optional, and can be selected by relevance to your program and your clients. These items include:
- Extended client demographic details such as Homeless indicator through to Ancestry.
- We have Client needs and circumstances so we are looking at the reason for seeking assistance and referrals on to other services.
Again as I’ve mentioned, all these items are optional.
So we will take a look at extended demographics.
- The recording of extended demographics is optional even if you are in the Partnership Approach
- It can be recorded where participating in the Partnership Approach as well.
- You would only need to record the extended demographics items appropriate to your client group or program activity. So if you are delivering an aged care delivery service and want to see if a client has support available to them when they get home, you might only need to record the Household composition of that client.
- Extended demographics will be valued by your organisation and the Department as adds to our picture around complexity of client needs. It provides a context of your client’s circumstances, what their situation their facing and where they’re coming from and also why they are accessing your services.
- What you record will be available to you in the Partnership Approach (reports) as long as you do opt in to the Partnership Approach.
So who might want to use these items and why? We’ve got just a few limited examples but we will go through them together so that you can get a better understanding. We’ve had feedback that the Homeless indicator is very relevant for programs that target homelessness like Reconnect or CHSP Assistance with Care and Housing. Household composition and having a better understanding of family groups may be of benefit to your organisation to programs such as family and relationship services.
Data items relating to Income regularity and amount could be beneficial to organisations delivering financial wellbeing programs designed to improve clients earning capacity. If your organisation is funded to deliver or has services that target domestic violence this may be a relevant field for you also. There has been some evidence to suggest that people often stay in with abusive partners due to financial dependency, so it could be a really good data item there to capture.
Visa type and ancestry are often relevant for Settlement Services who want a better understanding of their client backgrounds.
Client exit reason may be relevant when delivery CHSP or disability support services as often when a client leaves a service we get a better understanding of the success of their contact with a service.
Referral sources types and purpose can provide great insight into where your business is coming from. If clients are more often self-referred, this could demonstrate that your website is easy to use or your organisation is well known in your area. If your GP referrals are low you may need to get in contact with your local health care provider and let them know about the services that you deliver, so these referral types and services are very good.
Let’s now take a look at SCORE. SCORE stands for Standard Client Outcomes Reporting and it is a standardised way of reporting the types of outcomes clients are achieving through the accessing of programs that your organisation delivers.
So how does SCORE work? Well firstly you would need to determine the way an assessment will be made and we are going to have more on the different types of assessments methods very shortly. You will then need to select the relevant SCORE domain. Will it be under, Circumstances, Goals or Satisfaction for individuals or will it be a Community for a larger group.
The example that is shown on the screen is of the SCORE Goal domain and the intended outcome is for Changed knowledge and access to information.
You can also note by that example that is shown on the screen that it is an example of what we call a Likert scale so it has ratings from 1 to 5 that a client would then actually agree or disagree with a particular statement. So you would then record a pre SCORE at the beginning or a client’s engagement with a service or activity by selecting one of those numbers on the Likert scale of 1 to 5. At the end of a client’s engagement with a service or activity, you would record a post SCORE. If a client is attending a service or activity long term and you would like to measure their progress, you can record multiple post SCOREs periodically throughout their interaction with that service or activity.
Now the difference between the pre and post assessment allows us to look at the amount of change in the client’s outcome as a result of receiving and accessing the service from you.
Some key points on SCORE. So we did mention earlier that SCORE should be recorded for at least 50 per cent of your clients that access the services in any reporting period. Now we do find that in some instances, this may not be possible due to a number of reasons, so we do even say that if your organisation was able to capture 30 per cent - that is okay. However the higher amount of SCORE that you can record, the stronger and robust the data that you will see in return in your reports. At least 10 per cent of your clients will require a Satisfaction SCORE and only a post SCORE is required for this as clients cannot be pre satisfied.
You would then need to pick the domains that are most relevant to your client and the services that you are delivering. You can use a combination of domains or just one, again depending on the services that your client is accessing and what outcome your client is hoping to achieve.
There is no right or wrong way to move from the different SCORE scales. There’s going to be times when you will record a SCORE that shows that your client is moving backwards and that is okay, or that the same client may have differing outcomes for each case that they are associated with, that’s fine. What matters is that you are accurately recording your client’s outcomes at that point in time.
As mentioned at our previous slide SCORE has four domains that can be selected. We have Circumstances, Goals and Satisfaction for individual clients and then the Community SCORE for the larger client group such as Harmony Day or a community event BBQ, where it is impractical to record SCORE on every person attending.
Each of the SCORE groups target a specific area and a set of domains within it relating to possible client outcomes and you can see those there under the headings there of Circumstances, Goals, Satisfaction and Community, the different intended outcomes listed.
So the Department realises that while one domain may relate to services being delivered by one organisation, another may not. Now they’re all included because SCORE is used by a broad range of programs that are run by different funding agencies.
So your organisation should chose the relevant domain and its possible outcomes based on your client’s needs and what they are aiming to achieve or improve through contact with your service. We will be exploring some examples together shortly to actually give you an idea of which SCORE areas could be useful depending on the context.
So for those organisations that will be using the web-based portal, this is what the SCORE rating looks like. So to record a SCORE it is a simple as selecting the pre or post assessment phase selection and then you select the appropriate scale rating against the client domain. So the example that we have here is a SCORE domain for Circumstances and you can see that the intended outcomes that have been selected are Family Functioning at a scale of 2, Housing at a scale of 3 and Mental health and wellbeing at a scale of 3.
The pre and post SCORE screens look the same. Task card 12 and the e-Learning module number 14 entitled Add a SCORE assessment provides further instruction and information on how to record a SCORE.
So again this is what SCORE looks like in the portal, if you have your own system and it has been adapted it might look different. Organisations that are in the Partnership Approach have let us know that in terms of time, it takes more time to set up SCORE, to discuss and decide which domains to use and collect than actually the recording of the SCORE. Appendix B provides further guidance into the input of score into the system and this will assist your organisation in getting that conversation and decision moving.
So who can make a SCORE assessment? SCORE assessments can be determined in a number of ways and you can those on the screen. So you could use a combination of these suggestions, depending on the service that you are providing and what would work best for your clients. So a client can actually make a SCORE assessment, so some organisations use a printed SCORE matrix and provide it directly to their client to circle or mark, where they feel they are on the scale. Then they would return this to staff at the organisation who would then enter it into the Data Exchange at a later time.
Some organisations prefer their trained counsellors or practitioners to make a professional judgement as to their client’s outcomes. Others have staff discuss circumstances and goals with their clients and complete a joint assessment between practitioner and client. Many organisations also use clinical measurement tools to assess outcomes for clients. These could include for example the Kessler K-10.These can continue to be used and a similar result translated into SCORE. Or a practitioner can observe a client and record a SCORE based on what they see and hear from the client.
The key point is that SCORE is not so much a method of collecting client outcomes but recording these outcomes in a consistent way across all the programs that use the Data Exchange.
So SCORE is not a clinical evaluation tool and it may be used differently by different organisations. In the previous slide we discussed some organisations using clinical measurement tools such as Kessler K-10 to assess their client outcomes. Now a Translation Matrix has been developed by the Australian Institute of Family Studies or AIFS to translate the parameters of nine common assessment tools that are noted on the screen into SCORE. So for example, you may have a Kessler K-10 score for a client of 35, the Translation Matrix will assist in converting this rating to a SCORE rating. So to help with those nine ones that are seen on the screen we have the Translation Matrix which is available on the Data Exchange website. Now this document should be taken as a reference only. If your organisation is using an assessment tool that is not listed here you could refer to the Translation Matrix and see how these particular nine assessment and evaluation tools have been converted and you could have a discussion again within your organisation of having that translated into SCORE language.
Let’s now take a look at some examples on how to record SCORE. The first example that we are going to look at is Ned who is a grandfather and he has been told by his children that he needs to improve his parenting skills if he wants to continue to look after the grandkids. So Ned enrols into a parenting course, it’s a six week course as you can see on the screen there, there are six sessions, and the outcome that Ned is trying to achieve by attending this course is ‘Changed skills’.
So at the beginning of his attendance, he is provided with a quiz that he completes. Now he has made a genuine effort on this quiz and so the rating here that he has been given is a number 2 which is ‘Limited progress to date in achieving skills, goals but emerging engagement’ because he has made that genuine effort and he is also enrolled in the course. Over the course of Ned attending these weekly sessions, he hears from speakers about the relevant topics such as child nutrition, suitable child activities, etc. So on this final session Ned retakes the parenting quiz and this time he does much better. So he’s really benefited from attending this particular course and so the post SCORE rating is 4 of ‘Moderate progress to date in achieving skills and goals’ against the Changed skills intended outcome.
The second example that we will have a look at is from John and Sally, they’re having relationship issues and they have been fighting a lot, so they enrol in the Family and Relationship Counselling Service with the intended outcome to improve ‘Family Functioning through counselling’. So straight away what we can see here on the screen, is that there are three sessions involved in this particular activity and service that is provided by the organisation. So at the beginning of the attendance there at these sessions they’re both recorded with a pre SCORE and you notice that they are different individuals so it is where they are at that particular time that this pre SCORE is undertaken. Once they’ve attended the sessions and had the necessary discussions and assistance required; a post SCORE has been facilitated on that final session and so we can see that they have moved slightly against that rating against again the intended outcome of ‘Family Functioning’.
The final example that we will look at is about material assistance and we have Betty and her family who have fallen on hard times, having trouble paying bills and getting enough food on the table to feed the growing family. So they attend an organisation’s Emergency relief service and the intended outcome is to provide ‘Emergency relief to families in need’. So the difference we can see here with this one is that where the previous examples had a number of sessions, this particular example that Betty is accessing only has one session and we have two different intended outcomes. We have ‘Changed impact of immediate crisis and we have ‘Changed engagement with relevant support services’. A pre SCORE has been recorded for Betty of 1 for ‘Changed impact of immediate crisis’. The reason that she was attending this organisation was to obtain food parcels. Now that need had been met, so the post SCORE that was recorded what the outcome was achieved by Betty.
In relation to the ‘Changed engagement with relevant support services’, initially when Betty attended this particular service we have that pre SCORE that is recorded, however, during her time at this particular emergency relief service there were other services that were recommended to Betty and she advised that she would these offers up and felt a lot better from receiving this service there so a post SCORE was recorded with that amount there that you can see.
So as you can see from the three examples that we have provided, there are different ways that SCORE can be recorded against the different intended outcomes and again remembering that it’s an accurate picture of the period of time at the pre or post SCORE area that is recorded against the SCORE for them.
So we will now look at how you can opt in and join the Partnership Approach. The slide on the screen gives you the different steps that you need to take. One of the most important things and the first thing really is to have a discussion within your organisation as to which particular Extended data items would be beneficial for you to collect.
Also too, discuss about how you are going to record SCORE, if you are using an assessment tool how that can be translated into SCORE language and how you can actually collect pre and post SCOREs from your clients.
The next step that you would need to take is to update the Partnership flag in the Data Exchange web based portal and I will show you how to do that and it is a very easy step that your Organisation Administrator will need to undertake.
Then talk to your Grant Agreement Manager and let them know that you would like to be part of the Partnership Approach. You can either send them an email for official notification of this. Now what will happen is, they will insert a Partnership Clause into your grant agreement that will be a Notice of change, however they won’t send you out a Grant Agreement to you straight away to reduce red tape for everyone may wait until the agreement needs to be varied for another reason such as additional activities, extensions, etc. However you can start collecting and submitting the Partnership Approach data straight away and you don’t need to wait for that variation because that can even happen via email.
If your organisation chooses to opt out at a later date the clause can be removed via a variation. Again you would need to let your Grant Agreement Manager know and update you partnership flag, however the first step having that discussion within your organisation and decide which client or Extended data items will be collected and how you are going to start collecting and record that SCORE information.
Now for all organisations regardless of the upload method of your data, your Organisation administrator or Org administrator will need to go into the website and update the Partnership flag, so I will just take you through the steps on how to do that. So when they log into the web based portal, these that section of Manage Organisation that they would need to click in to. The Manage organisation screen will display as an example here is shown. They would then need to select the Edit organisation details button and the Edit organisation details screen will display. Now you will notice under the Organisation details heading, the Partnership Approach agreement buttons, so at the moment this one is selected or set at No so they will then select Yes and change that agreement, select Save, and when we go back to the Manage organisation screen we can see clearly that the Partnership Approach agreement has now changed from No to Yes.
So that’s at an organisational level you will also need to make that change at a program activity level too. So on the same screen as the Manage organisation level your Org administrator will just scroll down and come to the Program activities heading, select the relevant Program activity, select Edit program activity button, update the Partnership Approach button, select Save and it will then save and show the Partnership Approach agreement as Yes. More information on how this can be done, or step by step information can be found in Task card 4, Create and manage outlets, that is found on our, the Data Exchange website.
So your organisation can opt out of the Partnership Approach for an individual program activity level even if it has opted in at an organisation level. If your organisation delivers the same program activity at multiple outlets, each of these outlets must all opt in or out as participation cannot be assigned at the outlet level it is only at the program activity level. So to opt in you need to select Yes, or change from No to Yes for the Partnership Approach Agreement at the organisational level and also the program activity level.
So with SCORE, what’s in it for you and your organisation? By reporting SCORE you are able to measure the outcomes being achieved by your clients and you are able to see all of this via the SCORE report which is currently under development.
The Department wants to reward those organisations that have put in the time and effort taken to report and complete SCORE with this new report. You will be able to see where your programs are affecting the most change and see what services are successful and working and which services may need to be tweaked to deliver the outcomes that you wish to achieve. You will be able to see the average shifts in outcomes via the pre and post SCORES recorded and you will also be able to see the percentage of your clients that have SCORE outcomes reported.
What we have on screen here are the current reports available to those organisations that are participating in the Partnership Approach. So, on the left hand side we have the Service Footprint report and we also on the right hand side the Resource Planning report. Now straight away, you can see that they’re very colourful and they are also very dynamic because you can go and click on any of these areas and it drills you down to further information. So the Service Footprint report will provide information regarding outlet and client demographics within those outlets and going from a larger area or boundary area such as a SA3 right down to your local government area such as your suburb view.
Resource planning is very good for scheduling and planning for your business, it looks at the different cases and sessions that have been held over the reporting period, the different clients as well, that have attended those sessions, so very in good at determining staffing levels, what the different activities that are happening at different outlets and again you can drill down further into those graphs or images and it again takes you down into further information.
Other Partnership Approach reports that are currently under development are the Community Profile report. This report draws in data from other population data sets such as ABS, SEIFA and also Census data information. The feedback that we have been receiving is fantastic as it is in one place, whereas previously organisations have had to search for different reports and then put it all together. This is all in one place for you so it makes it a lot easier, and we are building and incorporating new different types of information in this report.
The next screen shot that you can see is of the Extended demographic report and it will provide you information and details regarding your client profiles, so their cultural and linguist diversity, their home situation, the reasons for seeking assistance that you can see here on the screen, and again all these reports are very interactive and dynamic, clicking on any section here and drill down into further information. We then have a program specific report – Standard report for CHSP. Now all the information that is being presented in these report for this information session, is all training data and not live data and we have an example here of the SCORE report and what it will look like.
I’m sure you’ll agree that the new suite of Partnership Approach reports is very exciting and is going to be very, very useful for your organisations in determining your client base, what your clients are achieving from accessing your services and also giving that complexity and background of where they are coming from and why they are accessing your services. So again we would encourage organisations to have that discussion as to which particular Extended data items would be very beneficial to start reporting on and how to capture that SCORE information.
That is the end of the information session today, further information on the Data Exchange can be obtained from the Data Exchange website. We have the Training resources tab that provides extensive instruction on the different processes and background information on the Data Exchange. We also have policy guidance materials such as the Protocols document, Appendix A and Appendix B which will be very useful for your organisation as well, also available there under the Policy Guidance tab, and there is a section if you do need to contact us with any further questions or information.
Thank you for viewing this Partnership Approach, SCORE and Reports information session.
This information session builds on existing knowledge and prior training to promote a deeper focus on the collection of Partnership Approach information.