If you are part of our organisational pilot of the Digital discovery tool, you will now have access to your data dashboard with visual results from your staff users. Guidance for accessing and reading your data visualisations can be found here.
There is also a collaborate webinar on Tuesday 17 April at 13:00 which will walk you through the process and help you to make use of your data. You can access the webinar live here, or after the event you can access the recording here.
The rest of this post is about how you might make use of the data in your organisation. Please remember that the data provided as part of the pilot is still in development. We are in the process of finding out what data is useful. You should not rely on these data visualisations as a definitive source of information about staff training needs.
Making use of your data
You may want to use the number of staff completions – possibly broken down by department – to compare the number of staff who have fully engaged with the number of staff you hoped to reach at the start of the project. Who has and who has not engaged? Do you have feedback from your engagement sessions or a follow-up process (e.g. focus group) to explain any differences? How might you encourage engagement from other groups of staff?
You could also compare the number of staff who have completed the general (‘all staff’) assessment with the number completing the specialist teaching assessment(s). How would you explain any differences? Again consult with your users: were teaching staff more motivated and satisfied by the role-specific assessment?
The ‘in progress’ data allows you to see if there is a significant drop-off as staff are going through an assessment. This is a figure Jisc is looking at closely, as the user experience needs to be easy and supportive – that is our responsibility. But if you find differences in the drop-off rate across different staff groups, could this be because of differences in the support you make available to them?
Scoring band data should be interpreted with great caution. Jisc is using this data to ensure that the questions we ask produce a reasonably even spread of medians across the different areas of digital capability. But this is a broad aspiration: it is inevitable that some areas will prove more challenging to users than others. Also, some areas are essential for all staff (such as digital wellbeing), while others such as information, media or data literacy are more important in different roles.
This is why all our feedback to individual users asks them to reflect on their role and its demands before deciding how to prioritise their next steps. It is also why you should not compare scoring bands across completely different areas of digital capability and conclude that your staff have a ‘deficit’ in one area as compared with another. If you want to make comparisons, look at overall sector scoring bands and compare with the relevant banding in your organisation. But even this should be done with great care, particularly if you have a low number of users overall or in one departmental group, as this will skew the results.
Scores are all self-assigned, and their purpose is to ensure that users get appropriate feedback. If staff believe that their scores are being used for another purpose, they may not answer questions honestly, and the value of the Digital discovery tool will be severely limited.
Jisc encourages you to use the Digital discovery tool to support a dialogue with staff about the training and development they need. The spread of scoring bands across different departments may encourage you to target training in specific areas towards specific groups of staff. Because of the caveats above, you should not do this without consulting with the staff involved. Where staff score lower than others in their sector, this is definitely a cue for you to investigate whether they would appreciate more training and support, but it is not a performance measure and should never be used as such.
Following up and closing the feedback loop
The information you gather from the Digital discovery tool can be used to start conversations:
- with HR and staff development about overall staff training and development needs;
- with teaching staff about their confidence with digital teaching, learning and assessment, and their further development needs;
- with IT and e-learning teams about support for specific systems and practices;
- with budget-holders about investing in staff development resources and in online services.
You should report back to your staff users about how you are using this data, and what you are doing to support them more effectively in the future.