Organisational data available

If you are part of our organisational pilot of the Digital discovery tool, you will now have access to your data dashboard with visual results from your staff users. Guidance for accessing and reading your data visualisations can be found here.

There is also a collaborate webinar on Tuesday 17 April at 13:00 which will walk you through the process and help you to make use of your data. You can access the webinar live here, or after the event you can access the recording here.

The rest of this post is about how you might make use of the data in your organisation. Please remember that the data provided as part of the pilot is still in development. We are in the process of finding out what data is useful. You should not rely on these data visualisations as a definitive source of information about staff training needs.

Making use of your data

You may want to use the number of staff completions – possibly broken down by department – to compare the number of staff who have fully engaged with the number of staff you hoped to reach at the start of the project. Who has and who has not engaged? Do you have feedback from your engagement sessions or a follow-up process (e.g. focus group) to explain any differences? How might you encourage engagement from other groups of staff?

You could also compare the number of staff who have completed the general (‘all staff’) assessment with the number completing the specialist teaching assessment(s). How would you explain any differences? Again consult with your users: were teaching staff more motivated and satisfied by the role-specific assessment?

The ‘in progress’ data allows you to see if there is a significant drop-off as staff are going through an assessment. This is a figure Jisc is looking at closely, as the user experience needs to be easy and supportive – that is our responsibility. But if you find differences in the drop-off rate across different staff groups, could this be because of differences in the support you make available to them?

Scoring band data should be interpreted with great caution. Jisc is using this data to ensure that the questions we ask produce a reasonably even spread of medians across the different areas of digital capability. But this is a broad aspiration: it is inevitable that some areas will prove more challenging to users than others. Also, some areas are essential for all staff (such as digital wellbeing), while others such as information, media or data literacy are more important in different roles.

This is why all our feedback to individual users asks them to reflect on their role and its demands before deciding how to prioritise their next steps. It is also why you should not compare scoring bands across completely different areas of digital capability and conclude that your staff have a ‘deficit’ in one area as compared with another. If you want to make comparisons, look at overall sector scoring bands and compare with the relevant banding in your organisation. But even this should be done with great care, particularly if you have a low number of users overall or in one departmental group, as this will skew the results.

Scores are all self-assigned, and their purpose is to ensure that users get appropriate feedback. If staff believe that their scores are being used for another purpose, they may not answer questions honestly, and the value of the Digital discovery tool will be severely limited.

Jisc encourages you to use the Digital discovery tool to support a dialogue with staff about the training and development they need. The spread of scoring bands across different departments may encourage you to target training in specific areas towards specific groups of staff. Because of the caveats above, you should not do this without consulting with the staff involved. Where staff score lower than others in their sector, this is definitely a cue for you to investigate whether they would appreciate more training and support, but it is not a performance measure and should never be used as such.

Following up and closing the feedback loop

The information you gather from the Digital discovery tool can be used to start conversations:

  • with HR and staff development about overall staff training and development needs;
  • with teaching staff about their confidence with digital teaching, learning and assessment, and their further development needs;
  • with IT and e-learning teams about support for specific systems and practices;
  • with budget-holders about investing in staff development resources and in online services.

You should report back to your staff users about how you are using this data, and what you are doing to support them more effectively in the future.

Using Discovery tool data to refine the questions and scoring

Thanks to the aggregate data we are getting from our first pilot users, we have been able to compare the median scores for each of the questions asked, and look at some other stats across the different assessments.

We were pleased to see from the first data returns that ‘depth’ and ‘breadth’ questions produce the medians we would expect, with one or two exceptions. We’ve worked on these outlying questions to make it a bit easier (or in one case a bit harder) to score in the middle range. This should bring the medians more into line with each other, making it easier and more valid to look across aggregate scores and compare areas of high and low self-assessment.

Median Question Scores - All capabilities

Median scores, ‘all staff’ assessment, snapshot from early March 2018: click for detail

There will always be some natural variation in average scores, because we are asking about different areas of practice, some of which will be more quickly adopted or more generally accomplished than others.

We were particularly pleased to find on testing that there is a positive correlation between confidence and responses to other questions in the same area (i.e. expertise and range). We would expect this, but it is good to have it confirmed. However, although there was a meaningful range of responses, almost no users were rating themselves less than averagely confident, so we are looking to adjust the scoring bands to reflect this. We don’t attach a great deal of weight to this question type, precisely because it is known that users tend to over-state their confidence, but is included to encourage reflection and a sense of personal responsibility.

You will see the impact of this work when we reach the mid-April review point, along with some further changes to the content and platform indicated by our user feedback. More about this below.

Scoring is designed to deliver appropriate feedback

As you see, we’re doing what we can to ensure that the scores individuals assign themselves are meaningful, so they allow relevant feedback to be delivered. The question types available don’t allow us to match selected items with feedback items (e.g. items not chosen in the grid or ‘breadth’ questions with ‘next steps’ suggestions in the personal report). This means relying on aggregate scores for each digital capability area. The pilot process is allowing us to find out how well the scoring process delivers feedback that users feel is right for them, and how the different areas relate to one another (or don’t!). However, the questions and scoring are not designed to provide accurate data to third parties about aptitude or performance. So scoring data, even at an aggregate level, should be treated with a great deal of caution. We are issuing new guidance on interpreting data returns very shortly.

radial

The radial diagram gives a quick overview of individual scores

The aim of the Digital discovery tool is developmental, so it’s clear what progress looks like and ‘gaming’ the scores is simple. Our contextualising information is designed to remove this temptation, by showing that the discovery process is for personal development and not for external scrutiny. Our feedback from staff in particular suggests that if there is any suggestion of external performance monitoring, they won’t engage or – if required to engage – they won’t answer honestly. Which of course will mean there is no useful information for anyone!

 

The ongoing evaluation process

evalform

Showing where to find the evaluation form on the dashboard

As well as examining user data, of course, we have access to the individual evaluation forms that (some) respondents fill out on completion.This is giving us some really useful insights into what works and what doesn’t.  However, at the moment we think the sample of respondents is weighted towards people who already know quite a lot about digital capability as a concept and a project. The views of people with a lot invested are really important to us. But we also need the feedback from naive users who may have a very different experience. Please encourage as many as possible of your users to complete this step. The evaluation form is available from a link on the user dashboard (see right).

Screen Shot 2018-04-01 at 22.01.53In addition we have taken a variety of expert views, and we are just about to launch a follow-up survey for organisational leads. This will ask you about what you have found beneficial about the project, what has supported you to implement it in your organisation, what you would change, and how you would prefer Jisc to take the Discovery tool project forward. Please look out for the next blog post and launch!

Resources in the Digital discovery tool

The Digital discovery tool provides links to a wide range of resources for each of the digital capability framework areas.

The platform delivers these resources in two ways.

Browse resources on your dashboard

When people log-in to the tool they are presented with a tailored welcome page/dashboard offering appropriate assessments for them based on the selections they make during log-in.   The dashboard also includes sets of resources for each of the six broad digital capability areas. You can scroll through these sets and browse the resources that we have mapped to these areas. We offer a brief description of the resource in this view.

resources-dashboard

Once you see a resource that looks interesting you can click on it to find out more. For each resource we have identified key audiences and level as appropriate and provide a brief description to help you decide how relevant it is to you. When you click on the URL in the resource page you will be taken directly to that resource outside of the discovery tool.

For some resources we offer suggested activities or reflections and a space to record them to save for the future.

resource-page

Find resources in your assessment report

When you complete an assessment, you receive a personal report which offers results, feedback and suggests some next steps that you could take. You are also offered links to selected resources for each area. These are offered in the same kind of scrolling list with a summary about the resource. When you print your assessment results report the resources are offered as a simple list of links so that you can revisit these at a time convenient to yourself.

Resource selection

Resources included in the discovery tool come from a wide range of publishers. They are checked for accuracy, relevance and quality. They are all free to use although some may require users to register.

These publishers include:

  • national or international bodies (such as Jisc, Nesta, HEFCE, SCONUL, EU bodies)
  • professional bodies (such as CILIP, AoC, UUK)
  • educational institution resources produced for staff or students but which could be of interest to a wide range of users.
  • individual academics who have set up websites or blogs
  • educational consultants or specialists who have websites or blogs
  • networks of educators or specialist collaborators (e.g. supporting citizenship, research, innovation)
  • wikipedia and wikiversity
  • commercial companies (such as Microsoft, Adobe, Google)

Jisc has been working closely with some publishers including the Microsoft educator community, and the Duke of York Inspiring Digital Enterprise Award (IDEA) to map their resources to the digital capabilities framework and include them within the tool. Jisc is also working with the subscription based online learning platform Lynda.com to map their resources to the framework.

Jisc is aware that many educational institutions subscribe to resource collections and may want the discovery tool to link out to them. This is something we are thinking about and hope to implement in the future.

Each resource included in the discovery tool is reviewed for relevance to the framework area, content and quality. Many of the resources also reflect the next steps suggestions.

Following feedback from our pilot phases we have attempted to limit the number of resources that are offered to prevent overload. The collection is not meant to be comprehensive – it has been selected to map to the digital capability framework, the questions and the feedback.

While we only have limited space, we are always looking for great new resources so please let us know if you can recommend one. Even if we can’t include it straight away we will review it for future use.

Resource description

We provide information to help you decide how relevant the resource might be for you. Each resource has a description of the aims and content.

We highlight if a resource is aimed at a specific audience, sector or level. Several resources are aimed at a specific audience but could also be of value to people in other sectors of with other roles. For example a resource aimed at students may be of value to a staff member if their capability levels are just developing in that area.

All the resources are mapped to the digital capability framework and to the different areas covered in the assessments. For example, the same resource may appear in the section about media literacy, or in the teacher assessment on creating learning resources.

Some of the resources have a very specific focus such as ‘managing your emails’ while others are broader and cover a range of digital literacies.

We have included a wide range of formats – from whole courses or sections of courses to downloadable learning resources. We have links to videos, websites, networks, screencasts, toolkits, reports and guides. We have included links to the Jisc guides as these often offer links to further resources. Some of the resources are in the pdf format which will require you to download a pdf reader such as Adobe Acrobat.

Resource management

Jisc has longstanding experience of managing resource collections and will be updating and maintaining this collection. This means that if you go back to an assessment report you may sometimes find different resources listed. Dead links will result in resources being removed from the collection. If you find any links that do not work please report it to us.

Piloting the Digital discovery tool with students

While our current pilot projects have been getting the Discovery tool into the hands of staff, we’ve been working behind the scenes on the student version. We’re pleased to say that this is user testing well, with students particularly keen on the detailed final report. We’ll be promoting this more positively to learners as the prize at the end of their journey. Meanwhile we’re making some final improvements to the content, thanks to all the feedback from users and experts.

All this means that we’re looking for existing pilot institutions that are keen to extend the experience to students. You can express an interest by completing this sign-up form, and you can read more about what’s involved below.

About the student Digital discovery tool

Screen Shot 2018-01-15 at 12.19.58

The student version is designed exactly like the staff version, as described in this blog post. So users answer questions of three types, received a detailed feedback report with suggested next steps, and links to resources.

The content is designed to be:

  • Practice based: users start with practical issues, and the language is designed to be accessible and familiar
  • Self-reported: we trust users to report on their own digital practices. We attach very little weight to self-reported confidence, but we do expect learners to report accurately on what they do in specific situations (depth), and on which digital activities they undertake routinely (breadth).
  • Nudges and tips: the questions are designed to get users thinking about new practices and ideas before they even get to their feedback reportScreen Shot 2018-03-12 at 22.34.12.
  • Generic: different subject areas present very different opportunities to develop digital skills – and make very different demands. We aim to recognise practices that have been gained on course (after all these make an important contribution to students’ digital capability!) but where possible we reference co-curricular activities that all students could access.

Student users will find only one assessment on their dashboard, unlike many staff who will find a role-specialised assessment alongside the generic assessment ‘for all’. Most of the elements in the student assessment are the same as in the staff generic assessment, mapped to the digital capabilities framework. But the content is adapted to be more relevant to students, and the resources they access are designed to be student-facing, even where they deal with many of the same issues.

The ‘learning’ element of the framework is split across two areas to reflect its importance to students. These are ‘preparing to learn‘ with digital tools (mainly issues around managing access, information, time and tasks), and ‘digital learning activities‘. There is also an additional element, ‘digital skills for work‘, that sits at the same level as ‘digital identity’ and ‘digital wellbeing’ in the framework, reflecting the importance of the future workplace in learners’ overall motivation to develop their digital skills.

The feedback encourages learners to think about which elements they want to develop further, based on their own course specialism and personal interests. Where they score low on issues such as digital identity that we know are critical, we prompt them to seek advice. So use of the discovery tool may lead to higher levels of uptake of other resources and opportunities – and we hope this is seen as a successful outcome!

Screen Shot 2018-03-12 at 22.36.05There is some minor variation between the versions for HE and FE students, but we have done our best to keep these to a minimum. Our research and consultations don’t suggest that sector is an important factor in discriminating the digital skills students have or need. However, we do recognise that students vary a great deal in their familiarity with the digital systems used in colleges and universities. So we’ve designed this assessment to be suitable for students that are some way into their learning career, right up to those preparing for work.

It is not intended for arriving or pre-arrival students. We are considering a special assessment  for students at this important transition, but there are some problems with developing this:

  1. These students vary much more in their experience of digital learning, so it is much harder to design content that is not too challenging (and off-putting) for some, while being too basic for others.
  2. We are concerned that organisations might see it as a substitute for preparing students effectively to study in digital settings – this is not a responsibility that can be delivered by a self-reflective tool.
  3. We have learned from students that the most important content of an induction or pre-induction ‘toolkit’ is institution-specific – depending on the specific systems and policies in place.

So at the moment our focus for arriving students is to work with Tracker users to design a digital induction ‘toolbag’. The ‘bag’ is simply a framework that colleges can use to determine for themselves – from their Tracker findings and other data – how they want arriving students to think about digital learning, and what ‘kit’ of skills, devices etc they will need. More of this over on the Tracker blog soon.

What the Digital discovery tool for students is not

As above, the Discovery tool is not an induction toolkit, or any kind of toolkit. It doesn’t deal with local systems and policies, which are critical to students becoming capable learners in your institution. It does prompt learners to think about a whole range of skills, including their general fluency and productivity with digital tools, which will support them to adopt new systems and practices while they are learning.

Screen Shot 2018-01-15 at 09.28.22The Discovery tool offers access to high quality, freely-available resources, in a way that encourages learners to try them. In future you may be able to point students to your own local resources as well. But it isn’t a course of study and there’s no guarantee that learners will follow up the suggestions or resources offered.

The scoring system is designed to ensure students get relevant feedback, and to motivate them to persevere to the report at the end. It has no objective meaning and should not be used for assessment, either formally or informally. We have deliberately designed the questions to be informative, so it’s always clear what ‘more advanced’ practice looks like. Users who want to gain an artificially high score can do so easily, but we don’t find this happening – so long as they see the development report as the prize, rather than the score itself.

About the pilot process

Just like the staff pilot, we’re looking for quality feedback at this stage. If you’d like to be part of the journey, we’d be delighted to have your support. You’ll need to complete this sign-up form before the end of 23rd March – it’s a simple expression of interest – after which we’ll notify participants and send out your log-in codes. Our Guidance has been updated to ensure it is also relevant to the student pilot, and you’ll have dedicated email support. Access will be open to students until the end of May 2018.

Because this is a pilot, we are still improving the content and still learning how best to introduce it to students to have the most positive outcomes. This means changes are likely. It also means we’ll ask you and your students to give feedback on your experiences, as with the staff pilot.

Join us now: complete the sign-up form.

 

Digital discovery tool launched today

Universities, colleges and independent providers that have signed up to pilot the Digital discovery tool will receive their access codes today. On this page you can learn more about the new Discovery tool, the Potential.ly platform, the different assessments available, and the guidance that will help you put it all into practice.

Where we are today

The open pilot is taking place in 101 organisations (57 HE, 35 FE and 9 ‘other’) between now and the end of May 2018. You can find out more about the pilot organisations and their different approaches in this blog post.

log-in screenThe version launched today:

  • is based on a new platform from Potential.ly
  • offers completely new, user-tested questions + feedback for staff
  • links to a host of new resources, all openly available
  • offers further specialist questions + feedback for staff with a teaching role (in HE or in FE and Skills)

The new platform

Potential.ly is working with Jisc on the development of the new platform for the Digital discovery tool. The Potential.ly team has experience of delivering an accurate personality indicator to help students understand their strengths and ‘stretch’ areas across twenty-three traits and to prepare for employment. Their platform offers a clear visual interface for the Digital discovery assessments and feedback report.

Digital capability resources are available through the dashboard in an attractive, accessible style. This screenshot shows the browse view. Answering the assessment questions creates a personalised report for each user, with recommended resources to follow up.

Screen Shot 2018-01-15 at 09.28.22

The new design

The Digital discovery tool is designed according to the following principles:

  • Practice based: users start with practical issues as a way in to digital capability thinking
  • Self-reported: we trust users to report on their own digital practices. The scoring-for-feedback system means it is pointless for users to over-rate themselves.
  • Nudges and tips: the questions are designed to get users thinking about new practices and ideas, before they read a word of their feedback report.

Broad relevance: we have tried to avoid referencing specific technologies or applications to make the content relevant across a wide range of roles and organisations. Sometimes we use familiar examples to illustrate what we mean by more general terms.Screen Shot 2018-01-15 at 12.19.58All users are offered an assessment called ‘digital capabilities for all’, based on the 15 elements (6 broad areas) of the Jisc Digital capability framework. There are very few differences in the questions for staff in different roles or sectors, and students answer many of the same questions too, though the feedback and resources they get are a bit different.

Some users are also offered a specialist assessment, depending on the role they choose when they sign in. At the moment we are offering additional question set for teaching staff – ‘digital capabilities for teaching’ – as this was the priority group identified in our pre-pilot consultations. We will shortly offer another specialist set for learners, and one for staff who undertake or support research. More may follow, depending on demand. Users can choose to complete only the general or only the specialised assessment, but they must complete all the questions in an assessment before they get the relevant report.

The questions

There are questions of three kinds.

confidence question

Confidence question: rate your confidence with a digital practice or skill, using a sliding scale. The opportunity for self-assessment triggers users to be reflective and helps them to feel in control of the process.

depth question

Depth question: select the one response out of four that best describes your approach to a digital task. This helps users identify their level of expertise and see how more expert practitioners behave in the same situation.

breadth question

Breadth question: select the digital activities you (can) do, from a grid of six. We have tuned these so most users will be able to select at least one, but it will be difficult to select all six.

At the moment we know that some elements are harder to score highly on than others. Once we have a large data set to play with, we will be able to adjust these differences. But it may just be the case that some areas of digital capability are more challenging than others…

The feedback

Once all the questions in an assessment have been completed, users receive a visualisation of their scores, and a feedback report. The report can be downloaded to read and reference in the user’s own time – alone or with a colleague, mentor or appraiser.

radial

report

The feedback report includes, for each element assessed:

  • Level: this is shown as one of ‘developing’, ‘capable’ or ‘proficient’. Some text explains what this means in each case.
  • Score: this shows clearly how the user’s responses have produced the level grading
  • Next steps: what people at this level could try next if they want to develop further
  • Resources: links to selected resources for exploration

report detailed

The resources

All the resources available through the Discovery tool – whether they are recommended in the user’s personal report, or browsed from the desktop – are freely available, quality assured, and tagged to different elements of the digital discovery framework.

Screen Shot 2018-01-15 at 10.42.39

Making it better

This is a pilot, which means we are still learning how the Digital discovery tool might be useful in practice, and making improvements to the content and interface. For example, there may be some changes to users’ visual experience during the next weeks and months.

  • End-users are asked to fill in a short feedback form once they have completed one or more assessments.
  • A smaller group of ‘pilot plus’ institutions are going through the process with additional interventions and monitoring from the Jisc Building digital capabilities team, to help us learn from them more intensively.
  • All institutional leads are being asked to fill in an evaluation form and to run a focus group with staff to explore the impacts and benefits of the project.

These interventions help us to improve the Discovery tool and the support we provide for digital capabilities more generally.

What next?

In another post we will explore how to understand and use the data returns to organisational leads. We are also developing, for launch in March 2018:

  • A version for students studying in HE institutions, and for students in FE and Skills
  • A prototype for the Building digital capability website, to bring all our digital capability services and resources together
  • Four institutional case study videos
  • A senior leaders’ briefing paper
  • A study into how HR departments are supporting the development of staff digital capabilities (see http://bit.ly/digcaphr for more details)

Key resources

Digital capability community continues to grow

The second network event for our digital capability community of practice took place in Birmingham on 30 November 2017 with around 100 participants from over 54 colleges and universities coming together to share practice, exchange ideas and work together. The presentations, resources and Periscope recordings are available from our event page.

The strong interest in this community of practice signifies the centrality of digital capabilities to all aspects of educational practice and a recognition that digital capabilities are not only vital for the employability and future career prospects of our students but also have the potential to enhance institutional reputations and aid organisational efficiency.

Sharing practice

The contributions made on the day, by our presenters and by collaborative engagement in workshop sessions were greatly valued.

“I wanted to hear examples of other institution’s approaches to developing digital capability, and there were plenty of examples.”

Developing a holistic institutional approach

Our keynote speaker, Karen Barton, director of learning and teaching at the University of Hertfordshire, shared their approach to developing a holistic institutional approach to digital capabilities development. The formation of a digital capability steering group has been a key enabler, engaging senior stakeholders and sponsoring wider participation with their teams.

The Jisc digital capabilities framework proved useful in getting dialogue going and helping others to get to grips with the language and vocabulary used to describe digital capabilities. Karen also talked about the University of Hertfordshire’s model for staff development at different stages of their careers and the work on mapping where digital capabilities fit into their broader CPD framework and learning landscape.

Ongoing work includes the establishment of a student experience academic research group with a sub group focusing on technology enhanced learning and exploring whether the academic CPD model can be applied to other role profiles.

Watch the Periscope recording of Karen’s session or view the slides on our event page. See also their institutional story on their participation in the first stage pilot for the discovery tool.

Community-led discussions

Group work at digital capability community of practice 301117

Participants at the first digital capability community of practice event in May 2017 requested time for community-led discussions and topics identified by those registered for the November 2017 event included:

  • Effective staff development strategies – how to upskill staff with digital capabilities
  • Developing organisational approaches to digital capability and getting buy-in from senior managers
  • Measuring the impact of initiatives, tools and strategies on staff/student capability
  • Student digital capability, embedding digital capabilities into the curriculum and student/staff partnerships

Facilitated by community members, participants were tasked with identifying critical issues and sharing experiences and solutions – the outputs are captured on our padlet. A variety of strategies were used – in one group, participants were challenged to come up with 20 ideas/potential solutions in just five minutes with most achieving the target before being further challenged to identify one thing they could action the following day. This proved a very effective way of moving from general discussion to action-focused solutions in a short period of time.

Strategies for engaging staff

Community members cited staff engagement in digital capabilities as one of their most critical issues and so the opportunity to hear from four community members on their differing approaches was informative and insightful.

Non Scantlebury and Jo Parker both shared innovative techniques they’d used to engage staff in conversations around digital capability. Non asked participants to share their favourite apps and reveal their digital superpowers mapped to the framework; Jo used the ‘love letters and break up letters’ approach which elicited deep and more emotive feedback about the digital discovery tool.

Randeep Sami and Delon Commosioung shared strategies and practical examples of how they are engaging staff in their respective colleges.  Randeep explored the concept of the digital classroom and shared details of their 21st century teaching programme; Delon outlined how working as part of the quality team has helped to position effective use of technology as integral to teaching, learning and assessment.

One of the highlights of the meeting was a series of five Pecha Kucha sessions from community members willing to share their experience, practice and strategies. These short seven-minute presentations shared journeys so far, outlined institutional approaches and transformative ambitions, bringing the day to a well-paced end.

Video recordings and presentations are available to view on our event page.

Looking ahead to 2018

Building digital capability project update

A lot has happened since the first digital capability community of practice event in May 2017:

  • Tabetha Newman96+ institutions have signed up to take part in the second phase of our pilot of the digital capability discovery tool which runs from December 2017 to May 2018.  See Helen’s Beetham’s blog post Digging deeper with the discovery tool which provides a useful analysis of the motivations and aims of those signed up to pilot the new tool. See also Helen Beetham and Tabetha Newman’s update on the digital discovery tool including a succinct and entertaining guide on the differences discovery tool and the student experience tracker – complete with appropriate hats!
  • New for 2018: senior leaders briefing and video case studies – Recognising the strategic importance of digital capabilities, Jisc will be producing a senior leaders briefing in March 2018 along with four institutional video case studies. A study of how HR departments are supporting the development of digital capabilities is also underway with a report and case studies available in April 2018 – see Lou McGill’s blog post for details of how you can take part.
  • Visioning the new building digital capability service – Jisc is also working on the development of a new web-based portal designed to provide organisations and individuals with clear routes through the wealth of information, support options and resources available to support digital capabilities development. Keep up-to-date by signing up to the digital capability mailing list and the project blog.  As the prototype of the digital capability service is being developed we are looking for volunteers to get involved in some short online user testing activities (30 minutes or less). If you would like to take part, please get in touch with Alicja Shah.
  • A series of training events and webinars on curriculum confidence, digital well-being and identity, and digital leadership is also running over the next few months.

Shaping the next agenda: your take-aways and thoughts for future events

While Jisc has founded this community, the focus is very much on building a sustainable network and in facilitating participants to share the collective wealth of experience. Feedback from the event is very positive and naturally reflects the different stages people are at in their own personal and institutional journeys.

Participants valued the opportunities to hear the developmental journeys of others and highlighted other areas they would like to see more focus on at future events.

“It was good to be able to discuss issues and ideas with like-minded people as a small group”

“It has given me some ideas to try out.”

Suggestions for the next event include creating time and space for:

  • Discussion on the changing landscape around learner expectations and needs, societal views on education and the effect of this disruption and how digital capabilities feature in this
  • Feedback and case studies from students and their experiences in digital capabilities development
  • Networking with colleagues

What did you take away from the event?

What would you like to see on the programme for the next event?

Use the comments below or share your thoughts via the digital capability mailing list.

Save the date

We are delighted to announce that the next digital capability community of practice event will be hosted by the University of Leicester on 22 May 2018.

Do join us for what promises to be another rich exchange of ideas, approaches, strategies and resources.

Digging deeper with the discovery tool

We now have almost 100 organisations signed up to pilot the new Digital capability discovery tool. The new platform and content will be available to trial with staff from January to May 2018, and a student version will also be available in the trial period.

If you’re not part of the pilot, you can still follow the progress of the project from this blog and on the new dedicated Discovery tool pages (launching early December).

I’ve been digging through the undergrowth of the sign-up data from all 100 pilot sites, trying to get a clearer picture of who is leading the project and what their motives are. This is a brief report from my explorations.

  1. Job roles

Lead contacts for the discovery tool have a wide and interesting range of job titles.

Sign up data job roles

Job roles – click for more detailed chart

The largest category work in Digital education/e-learning/TEL (39) followed by Education/learning without a specific digital component (18). A separate cluster can be defined as specialists in Digital, IT or information literacy (8). These job titles included ‘Head of Digital Capability’, ‘Manager, information and digital literacy’ and ‘Digital Skills training officer’. Library/Learning Resources specialists accounted for another 8 sign-ups, Student experience/student services and IT/tech for 7 each, and HR/organisational/staff development for 5. There were also 5 subject specialist staff, of whom 3 were in English – a slightly surprising result.

These totals suggest that the bias of intended use is strongly towards teaching staff and learners.

  1. Sector

Nearly 60% of sign-ups came from HE providers and 33% from FE. The ‘other’ responses (9) came from work-based, professional and adult learning, combined HE and FE institutions, and routes into HE.

Screen Shot 2017-11-14 at 10.58.48

  1. Reasons for using the Discovery Tool

Users selected a mean of 3.5 different responses, with almost all selecting the motive to ‘Help staff to improve their own digital capabilities’.  Eighty-three percent were using the discovery tool to support a strategic change agenda, and there were also high scores for identifying and monitoring staff digital capabilities overall, and for raising awareness.

Sign up data reasons

It’s interesting that 37% of respondents hope to use the discovery tool to ‘monitor individual staff’, a feature that is not offered currently. We included this option to assess whether our original aims – to produce a self-reflective tool – match with those of institutional leads. It is these tensions that our evaluation will have to explore in more detail (more under ‘next steps’ below).

  1. Current approaches to supporting staff

There were 104 responses to the question: ‘What approaches do you currently have in place to support staff with development of their digital capabilities?’

Sign up data support

  • 43 (41% of) respondents mentioned voluntary staff development, especially of academic or teaching staff.
  • 14 (13% of) respondents mentioned some form of mandatory support for staff i.e. at appraisal, PDR, or induction.
  • 27 (26% of) respondents said provision for staff was currently insufficient, declining or ‘ad hoc’: Institutional support for staff to develop digital capabilities was removed about 6 years ago’; ‘[Staff training] has been very stop, start in recent years due to changes in management’; ‘There is no overall institutional approach to digital capabilities.
  • 20 (19% of) respondents mentioned IT or similar training, and a further 13 (12%) mentioned TEL support
  • 20 (19% of) respondents mentioned online materials, of which 15 were using subscription services such as Lynda.com and 5 had developed their own
  • A sizeable number (14) had a specialist digital capability project under way, though often in the early stages.

Nine respondents mentioned the Jisc Framework or (in two cases) another digital capability framework. Frameworks were being used: to support curriculum review; in teaching staff CPD; to design training (with open badges linked to the Jisc framework); and to identify gaps in provision. Two institutions mentioned the earlier pilot of the discovery tool as a source of support, and three were using alternative self-assessment tools.

  1. Current collection of data about staff digital capabilities

Asked ‘Do you already collect any data about staff digital capabilities?’ 58 respondents out of 101 (57.4%) said ‘no’ or ‘not at present’ or equivalent.

Among those who responded ‘yes’ (or a qualified ‘yes’) a variety of processes were used. These included:

  • Anonymous surveys
  • General feedback from staff training
  • Feedback from appraisal or CPD processes
  • Data from staff uptake of training or online learning
  • Periodic TEL/T&L reviews (inst or dept level)
  • Use of the discovery tool (n=10 – though in some cases this was prospective only)
  • Use of the digital student tracker (n=1)
  • Teaching observations

 

  1. Numbers involved and approaches to engaging staff

Organisations had very diverse ambitions for the pilot, from user testing with 6 staff to a major roll-out in the thousands. Strategies for engaging staff were also very different in the different cases. There were 104 responses to this question, and a lot of overlap with responses to the previous question about staff support in general.

Some 27 respondents decided that communication was key, and described the channels and platforms they would use. Strategies for gaining attention included having senior managers initiate messages, engaging students to design arresting visual messages, and involving a professional promotions team.  Timing was sometimes carefully considered e.g. to coincide with another initiative, or to avoid peaks of workload. In addition, 9 respondents considered the content of communications and the majority of these planned to focus on the immediate benefits to end-users: the opportunity to reflect, develop confidence, find out more, get advice and feedback, identify existing strengths. Other incentives were digital badges (2), support for an HEA fellowship application, and chocolate!

In all, 46 out of 104 or over 40% proposed completion of the discovery tool in live, shared settings, either at existing events such as meetings or at specific events designed for that purpose. Although we have emphasised the personal nature of the discovery process, it may be that shared, live events of this kind will prove more effective at developing trust, and providing on-the-spot support at a moment when users are receptive to it. One said:

‘Institutions that took part in the previous pilot of this tool identified the value of providing face to face support possibly as a lunch time ‘Digital Capabilities’ drop in to help staff complete the tool, explain those questions that staff didn’t understand and to discuss digital capabilities.’

  1. Planned support for the Discovery Tool

Users selected a mode of 3 different kinds of support (mean 2.8) – so there is a clear understanding that users will have contextualised support of various kinds. All but 3 were planning to offer users the ‘opportunity to share and discuss their results. Almost as many proposed to offer ‘links to local resources/development opportunities, in addition to the resources provided as an outcome of the discovery process. Around half expected to offer support through a staff network or community of practice, and slightly fewer expected to offer accreditation or badging.

Less than 3% chose to tell us about ‘other’ forms of support, suggesting that the support activities we identified in the last pilot still cover most of the sensible options. The main form of support noted in the ‘other’ responses (not already captured in the closed options) was the offer of tailored training opportunities mapped to the discovery tool content.

What next?

Information from this report is being used to inform the guidance for pilots of the discovery tool, and to identify issues for further exploration. A following post will have more detail about both the guidance and how we will be evaluating the pilot.

An important issue for institutional leads to consider is how much they can expect – and what they would do with – fine-grained data about staff capabilities. Data such as staff take-up of training opportunities, the quality of VLE materials, or student assessments of their digital experience, may be more reliable than self-assessments by staff using the discovery tool. This information could be obtained using the digital experience tracker, also available from Jisc.

We know from the earlier evaluation that the discovery tool can help staff to become more aware of what confident digital practice looks like, and more self-directed in their professional learning. And organisations will be able to collect aggregate data about the number of staff who complete the discovery process, and other information to be determined during the pilot phase.

We will be exploring what data is really useful for planning interventions around staff digital capability, how can this be generated from the discovery tool, and how  it can be reported without compromising staff willingness to engage.

For more information about the discovery tool please contact digitalcapability@jisc.ac.uk.

Participate in our study into how HR departments support staff to develop their digital capability?

Would you like to contribute to a new Jisc study and shape future developments in this area?

Jisc want to find out what HR departments are doing to support staff in dealing with the challenges and making the most of the opportunities offered by technologies? We are also keen to find out how confident HR teams are of their own digital capabilities to support staff in their institutions?

We have just started our short study and hope to find out from HR staff how their activities link to institutional strategies and activities around digital capabilities.

By participating in the study HR staff will have the chance to inform future developments and to highlight the good practice that is already happening.

We have produced a short (5 minute) online survey to help us create a snapshot of current practice in the UK and to find out about HR staff levels of confidence around digital capability.

Please let your HR teams know about the study and ask them to complete the survey.

https://jisc-beta.onlinesurveys.ac.uk/hr-support-of-staff-digital-capabilites

Designing for digital capabilities in the curriculum: what’s new?

When we launched the digital capabilities community of practice, a whole number of people signed up for a session on developing the curriculum. I noted some of the ideas that came out of that session, and I’ve been using it as a handy reference ever since.

It’s no surprise that session was so popular – we’re in the business of learning and the only outcome that really matters is our learners being able to thrive in a digital world. And while they gain many valuable skills informally and outside of the curriculum, the evidence is that complex, specialised digital practices need the support of subject specialists – people who understand their value and can introduce them in a subject context. So: how can we embed those digital capabilities into courses of study, in ways that engage both students and staff? What have we learned from previous conversations and programmes?

Some principles

I’ve been running workshops on embedding digital capabilities into the curriculum for some years now, dating right back to a Jisc workshop series in 2011/12. If you’re interested in some of the resources Jisc has developed since then, there is a good summary of curriculum resources on the Design Studio from the Developing Digital Literacies programme, and the Curriculum Change section of the Jisc Guide to Developing Digital Literacies (2014) is a more up to date selection. There are a few principles I’ve learned over the years.

Digital capabilities are subject specialised. Even the use of generic tools such as a spreadsheet or annotation app are highly dependent on the task at hand. But we know that students really value subject specialist technologies such as data analysis software, design tools and digital instrumentation, and specialist resources such as e-journals, reference management software and subject-specialist networks.

Every student brings their own personal digital practices to their subject, just as they bring their own literacy and numeracy practices, and their own preferences for different media. This variety of digital skills, experiences and preferences can be treated as a resource – for example through group exercises that allow students to learn from one another, or by offering different routes to assessment. They can also be discussed openly, rather than letting students feel their digital practices are not approved or not relevant to effective learning.

Staff need to be confident in their subject, their teaching, and their digital practice as part of those other forms of expertise. Digital confidence is an important quality. Students need to feel that their preferred learning practices are being supported and developed, and that staff are up to date with their professional skills. But they don’t need staff to be creating amazing digital content, to be as proficient in media production as they are, or to be engaged in all the same social media.

Digital capability is not a separate aspect of learning but integral to being effective in a subject area, or a vocation or profession. Nor is it separate from other agendas such as employability, sustainability or internationalism. Our world is digital, and global issues have a digital aspect. So look for digital activities that are complex enough to address several agendas. Introduce approaches that are genuinely used by digital researchers or professionals, not for the sake of being digital, but for the sake of achieving meaningful outcomes.

So yes, think about learning outcomes (the big picture conversation about what learners need to know) before thinking about methods and means (the technologies learners need to encounter). Digital technologies are changing every subject we teach. There are new research questions and methods in scholarly subjects, new approaches and ethical issues to consider in professional subjects, whole new branches of knowledge and qualifications that did not exist fifteen years ago. Change in the subject of study is interesting – for staff as well as students. Think about how the digital world changes the purpose of the course, and you will naturally be led to interesting activities that involve digital technologies in a meaningful way.

Using the digital capability framework in the curriculum

Many curriculum teams are already referring to the Jisc Digital Capability Framework – and especially the learner profile – to support their thinking. This can be helpful, but the profile is both too generic for detailed planning, and too specialist to be easily used by teaching staff who do not have an e-learning or digital capabilities background. If you do want to use the Framework to support curriculum design, here are some suggestions.

Print

  1. Don’t start with ICT proficiency or productivity, which creates a technology focus too soon.
  2. Also, don’t start with digital identity and well-being. Many aspects of this will be defined at the organisational level e.g. in terms of graduate outcomes/attributes, learning contracts, fair use policies and so on. However, it can be helpful to consider any digital aspects of the overall course outcomes, and to have those in mind while focusing on the four areas of practice.
  3. Focus curriculum conversations on the four ‘situated practices’ in the centre of the framework. Of these, the most productive and straightforward conversations with teaching staff are usually in the area of digital creation, problem-solving and innovation. This is about how problems are raised and solved and new knowledge or artefacts are created by subject specialists (‘scholarship’ may be a useful term in academic subjects).
  4. Next, think about information, data and media. Your conversations about method should have already raised issues about how these are created, managed, shared, visualised and used. It can be helpful to involve specialist staff e.g. from the library, learning support, or research support to think about what information, data and media literacy mean in this curriculum. This is an important place to discuss critical thinking and judgement.
  5. The other two areas of practice – communication, collaboration and participation, and learning (development) – are more generic across different subject areas. They include issues such as how learners make notes, record their achievements, set goals, organise their time and tasks, collaborate and so on. But it is still important to ask where in this  course of study these digital aspects of learning will be practiced and progressed.
  6. Once learning activities have been proposed to reflect these four areas of practice, then we can ask what technologies learners will be using – their own, or the organisation’s – to support the practice. And at that point learners’ proficiency and productivity may need to be addressed. If there are subject specialist systems that learners need to master – e.g. for data analysis, design, project management, instrumentation, administration etc – they need time to do this, and they need regular opportunities to practice and review. If learners are using generic systems, they will have to learn the specialist ‘rules’ for using them in scholarly or professional settings, rather than in personal or social ones.
  7. Finally it’s important to consider learner differences. How will you support students whose general digital skills need improvement? Do you know where to signpost them and how to do this without embarrassment? Schedule in an activity early on that will allow students to identify for themselves if they are under-prepared. Also think about students whose general digital capability is higher than average. They will be frustrated if they have to sit through scheduled sessions that introduce skills and systems they already have or can quickly master on their own. Allow them to progress quickly using self-paced support, and think about how their skills could be used to everyone’s advantage.

What’s new from Jisc for curriculum design?

Because of the importance of curriculum design for the digital capabilities agenda, Jisc has been offering short sessions at ConnectMore events to explore the framework in the light of curriculum concerns.

ConnectMore 1Resources from those sessions are now available from the links below, as many people have been asking how they can embed these ideas into curriculum practice at their own institution.

Sarah Knight, Head of Change (Student Experience), has this to say:

ConnectMore 2What I valued from running the Curriculum confidence workshops was the opportunity for us to re-focus on learning. Having conversations with staff about what they care about – their students and how best they can support them to acquire the knowledge and skills required to follow their career pathway. We are often led to focus on the technology rather than on the principles of what makes an effective learning and assessment experience. So for me its back to first principles and then seeing how we can utilise the affordances of technology to add the enhancements in order to prepare our students for a digital workplace.

Recognising that those sessions were not long or detailed enough for most participants, Jisc is now offering a one-day workshop entirely devoted to curriculum design for digital capabilities. Curriculum confidence: designing for digital capabilities in the curriculum will be running initially on 16 November 2017 in Birmingham: more details and a sign-up form are here. If you  join us you can be sure of opportunities to share ideas and practices with experienced curriculum staff from other universities and colleges, as well as an introduction to all the Jisc resources and current development projects.

Curriculum confidence resources mentioned in this post

Current Jisc resources:

Earlier resources (collated links):

How are HR departments supporting staff digital capability?

We are about to start a short review of how Human Resources (HR)  departments in educational institutions support staff with the development of their digital capability.

HR departments in universities and colleges increasingly recognise the need to contribute towards supporting staff to respond to challenges and opportunities offered by technologies. In 2016 their annual conference focused on the  The Changing Face of Work where they explored ‘How can we best equip our organisations to respond to our increasingly digital world and to meet the needs of our Generation Z employees?’  Helen Beetham presented at the conference about the Jisc digital capability work.

HR teams have a range of roles that require an understanding of the impact of technologies on staff:

  • recruitment
  • induction
  • staff health and wellbeing
  • conflict management
  • CPD support

HR staff also need to effectively use technologies and tools to deliver services. They need to ensure that their own digital capabilities are sufficient to make the best of available data and metrics and be aware of how social media and other technologies can impact on the changing relationships between students and staff. It is timely to find out how far HR departments are engaging with these challenges and to identify new and interesting practice.

Organisational approaches to developing digital capabilities

Jisc has been working with educational institutions to consider organisational approaches to developing digital capabilities and produced a supporting guide earlier this year. We hope that this work will build on those relationships we have already established with some HR teams and help us generate new links.

The work will review practice of HR departments in colleges and appropriate professional associations, and will also include an exploration of the role of trade unions in digital capability discussions and identify opportunities for developing strategies for their involvement.

Contribute to this review

The review will be carried out by Lou McGill and Tim Gray. If you are interested in contributing to this study, or know someone who may be interested, please contact Lou McGill. We will be launching an online survey soon to capture a snapshot of current practice.