Networked
 Learning Environments 		University logo

TLTP3-86: Facilitated Network Learning in Medicine and Health Sciences (Networked Learning Environments)

Project Plan

Version

TLTP3/86/02/7-Sep-98

Project Brief

To disseminate communication and information technology (C&IT) based approaches developed to facilitate, manage and support the interactive learning of medical and health care science students based in sites remote from the main university campus.

Objectives

  1. To disseminate a C&IT based approach to the provision of a networked learning environment (NLE), first to the consortium partners and then more widely to other interested HEIs. This approach, implemented recently to support medical students in the regionally-based Faculty of Medicine, University of Newcastle, is underpinned by a suite of C&IT applications developed specifically for the purpose.
  2. To implement a C&IT strategy in support of distance learning in undergraduate medicine and the biomedical sciences in the North of England (Universities of Newcastle and Durham).
  3. To build upon existing systems and C&IT tools to improve and extend the range of curriculum support services delivered to increasing numbers of undergraduate students and academic/academic support staff based both on campus and in remote sites (University of Newcastle).
  4. Using the University of Nottingham as a ‘mirror’ site, to evaluate the ‘transportability’ of the C&IT strategy to another region (Universities of Newcastle and Nottingham).
  5. To evaluate the acceptability, effectiveness and efficiency of the approach to students, academic teaching staff, and the course management and administrative support teams (Universities of Newcastle, Nottingham and Durham).
  6. To extend this C&IT strategy to include other programmes in the health care sciences in the region (Universities of Newcastle and Northumbria).
  7. To evaluate the potential use of the strategy for supporting students in other programmes/disciplines with needs and constraints similar to those that apply to medicine and health sciences (Universities of Newcastle, Durham, Northumbria and Nottingham).

Draft Checklist of Main Tasks (see also Appendix 1 - Gantt Chart)

Establish Functional Groups, Project Teams and Communication Channels

Meetings and Formal Communication

Complete Required Documentation

Evaluation Strategy — Evaluation and Analysis

Undertake Shared Dissemination Tasks

Develop Internet Version of the Networked Learning Environment (Lead Site)

Identify and Review Processes and Tools Appropriate for use by Partner Sites

Adapt NLE to Requirements of Nottingham

Adapt NLE to Requirements of Durham

Adapt NLE to Requirements of Northumbria

Disseminate the NLE to Other HEIs

Risk Analysis

Risk

Probability

Impact

Outcome

Minimising

Initially there are insufficient number of delivery systems with sufficient power to deliver NLE, and insufficient server power to support users of NLE.

3

4

12

Consortium works to identify weaknesses

Local infrastructure fails at crucial times, such as local networks and DNS overload during induction week.

3

4

12

Communicate with other local support services

Unable to disseminate NLE to medicine in another region, or biomedical and allied health sciences in this region. Failure to generate the required culture change and uptake of the NLE among the majority of staff and students. The requirements for printed copy still drives (dictates) the form of electronic delivery.

2

5

10

To overcome this using good dissemination activities is a key purpose of the project

Communication breakdown within the consortium and between the consortium and the wider HEI community, making it more difficult for non-funded members and others to take up all or some of the NLE.

2

5

10

Focus on communication with User Group

Some aspects, such as local student management and administration support systems, are not sufficiently generic to make them easily customisable.

2

4

8

Give good examples and documentation
         

 

Success Factors

Success Factor

Maximising Actions

Disciplines other than medicine, biomedical or allied health sciences are able to implement some or all aspects of this project (e.g. Law)

Early involvement of stakeholders in other disciplines in order to disseminate experiences and methodologies

Additional (interactive) teaching materials, or knowledge of methods of presenting them, are contributed to the NLE by non-funded members and the User Group

Encourage non-funded members to take up the NLE tools and contribute teaching materials which they have developed or knowledge of best use of third party products

Involvement of Publishers and Third Party Software Producers who work with NLE ensure that their products operate over campus networks - influences current methods of publishing

Demonstrate the NLE to Publishers and Third Party Software Producers and explain the use of the network in order to encourage them to produce or make available products in a networkable form.

Use of the NLE is taken up by a high percentage of staff at each site

Develop ways of involving all staff early in the production, adaptation and utilisation of the NLE.

Recommended by the GMC to Medical Schools in the UK (use of a NLE to support medical education becomes a ‘standard’)

Demonstrate that the NLE is of benefit to students and staff in maintaining educational standards in respect to changing geography of medical education.

National and International support for (and contributions to) the project

Make access to the User Group as wide as possible and encourage membership locally and abroad

Successful adoption of products by commercial interests (e.g. software houses build similar functionality into their products)

Demonstrate NLE to commercial interests with a view to their adoption of aspects of the environment.
   

 

 

TLTP3-86 Stakeholder Analysis

Stakeholder Groups (to be completed)

Probability

Impact

Evaluation Methods and Sampling Strategy

Project Members

      

Steering Committee

 		   

Interview and discussion (meeting minutes) - all

Project Manager

 		   

Documentation - all

Management Group

 		   

Focus group with MG + questionnaire - all

Project Teams

 		   

Focus group with PTs + questionnaire - all

Individuals in Project Teams

 		   

Interview and discussion, questionnaire - all including Project Officers

Students

      

Students on courses supported by a NLE

      

Mostly on-campus (phase 1 of course)

 		   

Module questionnaire*, logging, tests and questionnaires during timetabled teaching - all; random sample; random sample of sub-groups (e.g. sub-groups identified by user logs)

Mostly off-campus (phase 2 of course)

 		   

Logging, questionnaire, interview, focus groups - all; random sample; random sample of sub-groups (e.g. sub-groups identified by hospital location)

IT competent

 		   

Logging, on-line questionnaire - random sample identified by previous questionnaire

IT ‘naïve’

 		   

Logging, interview, paper questionnaire - random sample identified by previous questionnaire

Other students in institutions with NLE available to support some courses

    

Questionnaire? - random sample

Students in institutions without a NLE

    

Questionnaire? - to be decided

Staff

      

Staff delivering courses supported by a NLE:

      

Subject specialists

 		   

Logging, interview, on-line/paper questionnaire - all; selected representatives of particular types of specialist

Academic colleagues

 		   

Logging, interview, on-line questionnaire - all; random sample

Management, administration and clerical staff in faculties and departments which are implementing a NLE

 		   

Logging, interview, on-line/paper questionnaire - all; selected representatives (e.g. those with financial accountability)

Other staff in the institution

      

University computer service staff

 		   

Interview, on-line/paper questionnaire - random sample; selected representatives

Central administration (Management Information Systems) staff

 		   

Interview, on-line/paper questionnaire - random sample; selected representatives

Library staff

 		   

Interview, on-line/paper questionnaire - random sample; selected representatives

Academic Quality Enhancement and Staff Development

 		   

Interview, on-line/paper questionnaire - random sample; selected representatives

Other management and administration staff at consortium sites

 		   

Interview, on-line/paper questionnaire - random sample

Staff in universities which become non-funded members of the consortium

    

Logging, interview, on-line/paper questionnaire - to be decided

HEI staff in the User Group

    

Interview, on-line questionnaire - all

HEI staff in the Advisory Group

    

Interview, on-line questionnaire - all

Other Associated Interests

      

External project evaluators — appointed by project

    

Interview, focus group (meeting) - all

Evaluation team - Tavistock

    

Interview, focus group (meeting) - representatives

TLTP National Co-ordinating Team

    

Feedback from reports and informal contact - representatives

Other TLTP projects

    

Informal contact - representatives

Funding bodies

    

?

*As the NLE will form a part of the curriculum it can be evaluated along with the curriculum with respect to teaching quality assessment.

TLTP3-86: Facilitated Network Learning in Medicine and Health Sciences (Networked Learning Environments)

Evaluation Plan

Version

TLTP3/86/02/7-Sep-98

Areas of Innovation

Introduction

The provision of a low-maintenance networked learning environment (NLE) in medicine, biomedical and allied health sciences will support ‘guided self-learning’ among students located both on-campus and at distant sites. The NLE will be adapted to be delivered over the Internet and the technology and methodology will be disseminated using case studies of medicine in another region, and biomedical and allied health sciences in the North East of England (see Appendix 1 to the Evaluation Plan). The NLE may provide access to managed course documentation, information, communication and administration tools, and actual teaching materials placed in the context of the curriculum and the learning objectives. Student use of the NLE is anticipated to be as and when required, after an initial lesson to introduce it.

Evaluation will focus on documenting the processes and technical issues, and on uptake of the NLE by staff and students. It will explore reasons for uptake at an institutional level (e.g. quality issues, cost effectiveness, accessibility to the NLE) and at the individual level (e.g. personal attitude to C&IT), and document the characteristics of, and processes used in, the institutions in which they are able to be effectively embedded and their associated clinical teaching sites. Quality will be investigated in accordance with guidelines for teaching quality assessment categories and measures. It is anticipated that one or more aspects of the NLE will be adopted by other HEIs who will make up the User Group, and evaluation will investigate the reasons for uptake at non-consortium sites.

Extensive evaluation opportunities have been identified (see Appendix 1 to the Evaluation Plan) and it is anticipated that new opportunities and evaluation methodologies will become available during the lifetime of the project. Resources are limited therefore evaluation will focus on investigating the areas of innovation using the most economical evaluation methods appropriate to the task.

Innovation provided by the implementation and dissemination of an Internet-based Networked Learning Environment in Medicine, Biomedical and Allied Health Sciences

Define and provide examples of a process for implementing a cost-effective networked learning environment which supports medical, biomedical and allied health sciences students in order to maintain the quality of the educational experience with respect to changes in the delivery of health education:

Evaluation Methods and Outcomes

See Stakeholder Groups for details of groups to be evaluated and sampling strategies, and Appendix 3 for a list of evaluation questions which might be asked in relation to this project. Evaluation methods need to be responsive to changing circumstances and the needs of each site implementing the NLE.

Documentation and Case Studies

The major outcome of evaluation will be the production of documentation which describes the processes used in establishing and disseminating a NLE, with case studies in medicine, biomedical and allied health sciences. This documentation will reflect evaluative processes of formative (qualitative) evaluation of requirements, implementation, reflection, refinement and management documentation.

The resulting documentation will detail mechanisms for implementing a NLE, such as technical support; adaptation of existing or creation of new documentation and course materials; information technology awareness and ability among staff and students; cultural and attitudinal developments and the processes involved in bringing these about. It will also specify mechanisms for supporting the educational requirements of students on a regional basis, where educational responsibility is shared between, for example, the Medical Schools and the NHS, is facilitated by the NLE.

Social Survey Techniques: Qualitative Questionnaires, Focus Groups and Interviews

Questionnaires and focus groups will be used in order to inform the dissemination process in order to maximise the utility of interfaces and adapt NLE to partner requirements. Questionnaires can be disseminated on-line (e.g. email or WWW), or on paper.

Questionnaires and focus groups, face-to-face and telephone interviews will be used to obtain feedback about IT knowledge, the utility of the environment and attitudes to the NLE and communications and information technology.

Direct Survey Techniques: Quantitative Usage Logs and Assessment Grades

Information can be derived from the logs from WWW server and databases in order to monitor uptake of NLE by staff and students, according to location. These can be compared over time, in order to illustrate patterns of usage and interface issues (Quentin-Baxter, 1998; Appendix 4). Knowledge can be assessed using on-line tests. In-course assessment grades will be obtained in order to investigate any correlation of in-course assessment with uptake of NLE, attitudes to communication and information technology and other moderating variables (depending on available resources).

Identification of Evaluation Groups and Sampling Strategy

Stakeholders will be identified for each site (see example in Stakeholder Groups above) and will be targeted during the evaluation process to provide formative, quantitative and summative feedback regarding the impact of the NLE. Different evaluation strategies will be used depending on the stakeholder group and the purpose of the evaluation, such as use of electronically recorded usage logs, small focus groups and interviews to identify interface and systems design issues, and questionnaires and tests to assess all new first year students’ knowledge of and attitude to communication and information technology. Some stakeholders will be chosen at random from specific sub-groups which can be identified by the usage logs (e.g. samples of staff or students who have: a) never used the NLE; b) used the NLE a little; and c) used the NLE a lot).

Motivating students to provide evaluation feedback has presented difficulties in the past, therefore sampling and one-to-one interviews or telephone interviews will be utilised for formative feedback. The use of on-line questionnaires and feedback forms can generate automatic reminders but response rates to this technique are yet to be established. Specific questions arising about the NLE can be embedded in regular module evaluation for teaching quality assessment processes. Electronic usage logs are likely to provide a large amount of quantitative data which can be interpreted on its own, or in conjunction with the outcomes of other evaluation strategies.

For details of the evaluation of specific stakeholder groups and timing of feedback please see the Project Plan.

Evaluation Plan

Identify stakeholder groups and sources of information about individuals in stakeholder groups at Newcastle, Nottingham, Durham and Northumbria (see also Stakeholder Groups, Evaluation Methods and Sampling Strategy above). It is likely that this Plan will be expanded and may change as the project gets underway.

Survey project members and senior management at each site in order to establish:

Survey academic, administrative and management staff at each site in order to establish:

Survey students at each site in order to establish: