1. Document title: Jisc PotsPan Project Final Report
Last updated: February 2013
Project Document Cover Sheet
PotsPan Project Final Report
Project Information
Project Acronym PotsPan
Project Title Pilot of the SWANI Project Administrative Network
Start Date 1st May 1012 End Date 30th April 2013
Lead Institution Coleg Sir Gâr
Project Director Dave Howells
Project Manager & Professor Tony Toole
contact details tony.toole@e-college.ac
07964894790
Coleg Sir Gâr, Graig Campus, Llanelli SA15 4DN
Partner Institutions Pembrokeshire College, Swansea Metropolitan, JISC RSC Wales
Project Web URL Project Website: http://potspan.pbworks.com/
Project Blog: http://potspan.blogspot.co.uk/
Programme Name JISC e-Learning Embedding Benefits
(and number)
Programme Manager Rob Englebright
Document Name
Document Final Report
Title
Reporting May 2012 – April 2013
Period
Author(s) Tony Toole, Project Manager
& project
role
Date February 2013 Filename PotsPan Final Report
URL http://potspan.pbworks.com/w/file/63770491/PotsPan%20Final%20Report.docx
Access
Document History
Version Date Comments
v1 19/02/2013 First Draft
1

2. JISC PotsPan Project Final Report
JISC Project Final Report
Project name: Pilot of the SWANI Project administrative network (PotsPan)
Author: Tony Toole
Contact: tony.toole@e-college.ac
Date: February 2013
Contents
Acknowledgements 2
1. Executive Summary 3
2. Background 4
3. Aims and Objectives 4
4. Methodology 4
5. Implementation 5
6. Outputs and Results 5
Case Study 1: Coleg Sir Gâr 6
Case Study 2: Pembrokeshire College 7
Case Study 3: Swansea Metropolitan 8
7. Outcomes 9
8. Conclusions and Recommendations 9
References and Appendices
Acknowledgements
The PotsPan Project was funded by JISC as part of the e-Learning Embedding Benefits
Programme. The project team would like to thank Rob Englebright, the Programme
Manager, and staff at RSC Wales for their support and assistance. We would also like to
thank all those in the WBL teams in the partner institutions and in particular, the excellent
work by Lewis Jones who built the digitised WBL administration system.
2

3. JISC PotsPan Project Final Report
1. Executive Summary
The PotsPan (Pilot of the Swani Project administrative network) project was funded as part
of the Jisc e-Learning Embedding Benefits Programme. The goal was to take the outcomes
of the successful Swani (Secure Work-based learning through Networked Infrastructure)
concept-proving project completed by Coleg Sir Gâr in 2012 and to pilot it more broadly
across the regional partnership.
The Jisc Swani Project created a digitised administrative and management documentation
system for Work Based Learning in Wales that included a digital signature system that
satisfied the audit requirements of the EU and the Welsh Government.
The PotsPan project piloted the system with two of the partners in the Skills Academy Wales
South Wales (SAWSW) WBL consortium and also evaluated the approach in a partner
University for potential use in the administration of a new distance learning WBL programme.
The purpose of the PotsPan project was to take what was a successful concept proving
project and develop it into a practical digitised WBL administrative system that would not
only satisfy the needs of the SWASW partnership, but would also be of value to other
institutions addressing similar issues.
The broad aim of the PotsPan project was to implement the digitised online WBL
administrative system, created by the Swani project. The key objectives were to:
Refine, update and test the WBL document management system to ensure it met the
efficiency, usability and data consistency design requirements;
Test and evaluate the system with cohorts of work based learners in the SAWSW
partnership;
Explore the use of digital pens and electronic signatures as part of the WBL
programme audit arrangements;
Contribute to the European debate about digitised document management and the
use of electronic signatures for audit purposes.
The outcomes of the PotsPan project led the project team to conclude that the work had
confirmed the main benefits of administrative system digitisation:
Digitisation of the administrative system eliminated the need for the repeated entry of
core data and ensured data consistency;
Digitisation made the entire administrative system available to programme managers
at all times and from any location;
The system delivered significant operational efficiencies, mainly from the avoidance
of physically transporting learner files between different geographical locations.
The overall conclusion was that the PotsPan project had achieved its primary goal of
exploiting the benefits of the original Swani project and demonstrating its value and viability
in the practical administration and management of the work-based learning programme in
South West Wales. Furthermore, it was felt that the benefits were transferable to other
programmes with similar needs and hence contributed positively to the Jisc e-Learning
Programme Embedding Benefits objectives.
3

4. JISC PotsPan Project Final Report
2. Background
The JISC SWANI Project1 successfully created a digitised administrative and management
documentation system for Work Based Learning in Wales that included a digital signature
system that satisfied the audit requirements of the EU and the Welsh Government.
The PotsPan project piloted the system with two of the partners in the Skills Academy Wales
South Wales WBL consortium and also evaluated the approach in a partner University for
potential use in the administration of a new distance learning WBL programme.
The purpose of the PotsPan project was to take what was a successful concept proving
project and develop it into a practical digitised WBL administrative system that would not
only satisfy the needs of the SWASW partnership, but would also be of value to other
institutions addressing similar issues. In that context it directly contributed to the objectives
of the e-Learning Embedding Benefits programme.
3. Aims and Objectives
The broad aim of the PotsPan project was to implement the digitised online WBL
administrative system, created by the Swani project. The key objectives were to:
Refine, update and test the WBL document management system to ensure it met the
efficiency, usability and data consistency design requirements;
Test and evaluate the system with cohorts of work based learners in the SAWSW
partnership;
Explore the use of digital pens and electronic signatures as part of the WBL
programme audit arrangements;
Contribute to the European debate about digitised document management and the
use of electronic signatures for audit purposes.
4. Methodology
The PotsPan project methodology followed a straightforward testing and evaluation process
that began with a period of system refinement at Coleg Sir Gâr where the document
management system was hosted. The web designer and the project manager worked
together to test each of the system functions and the usability of each of the forms, including
how effectively they shared common data fields.
Workshops were held at both Coleg Sir Gâr and Pembrokeshire College with the WBL
teams that would be testing the system in the field. Each team selected an appropriate
cohort of work based learners and completed a testing and evaluation exercise that involved
completing all the digitised versions of the administrative documents in parallel with
completing the existing paper based versions. Each exercise concluded with a comparison
of the two approached with the aim of judging the relative advantages in terms of efficiency
and effectiveness. These comparisons would be presented in the form of case studies.
An additional exercise was carried out at Swansea Metropolitan where the use of electronic
signatures was explored. It was felt that this solution for document verification would be
eventually accepted by the EU for audit purposes, but that was not the case currently. The
exercise at Swansea Met, being unconstrained by EU rules, was designed to take that
agenda forward by testing and evaluating current electronic signature technologies and their
appropriateness for educational administration, particularly in the distance learning context.
1
http://www.jisc.ac.uk/whatwedo/programmes/elearning/swaniltig/swani.aspx
4

5. JISC PotsPan Project Final Report
5. Implementation
Following the usability testing exercise at Coleg Sir Gâr, the two partner cohorts for the WBL
implementation testing and evaluation were chosen and the process begun. The paper
based administration forms were completed as normal and were then transcribed onto the
digital system. The ease with which this was completed was noted, as was the potential
saving in time with the sharing of common data fields.
A perception at the outset was that there were administrative processes, particularly with the
registration of new learners, for example, where multiple forms were completed by hand that
contained the same information in terms of learner name, address, NI number etc. An
anticipated benefit was that the digitised system would only require these data fields to be
entered once, would ensure data consistency and, where editing was needed at a later date,
the edits would automatically update all forms that used that data.
The testing of electronic signature usage at Swansea Metropolitan was focussed on the
development of a distance learning version of a WBL programme delivered to the armed
forces in Germany. Of particular interest was the use of such signatures to verify the
authenticity of written assessment materials submitted online.
The outcomes of the testing were recorded using a common case study template for ease of
comparison. The outcomes were also shared on the project blog and wiki and were
presented for wider dissemination at ALT-C and at the JISC Online Conference 2012.
6. Outputs and Results
The Outputs and Results for the JISC PotsPan project included an updated and tested WBL
Document Management System, a series of three Case Studies that detail the evaluation
and testing of that system, and a project Blog that provides a narrative account of how the
project evolved. The final output was this report that provides an overall summary of
activities, achievements and deliverables.
The primary deliverable was the JOOMLA based document management system that was
originally developed during the Swani project. As a result of the PotsPan testing and
evaluation activities the presentation of the documents was refined and the efficiency and
consistency of data sharing was improved. The system can be viewed at
http://sawsw.colegsirgar.ac.uk/.
Examples of the documents and the way they are presented are shown on the project blog
at http://potspan.blogspot.co.uk/ and are available for download from the project Wiki at
http://potspan.pbworks.com.
There were three structured testing and evaluation exercises carried out as part of the
PotsPan project. Two of them tested the document management system, and the use of the
digital pens for audit purposes, with selected WBL cohorts at two of the partner institutions.
The third exercise explored the use of electronically signed documents, rather than digital
pens, in the administration of a work based distance learning programme at a third partner
institution.
The outcomes are presented in the form of three case studies. Single page summaries are
included below and the full case studies can be downloaded from the project Wiki.
5

6. JISC PotsPan Project Final Report
Case Study 1: Coleg Sir Gâr
The evaluation at Coleg Sir Gâr involved a group of modern apprentices at the TATA steel
plant in Llanelli and the digitised administrative system was carried out in parallel with the
existing paper based system and the outcomes compared.
Summary
{to be added}
Conclusions
{to be added}
Full Report
The full case study report can be viewed at: {to be added}
6

7. JISC PotsPan Project Final Report
Case Study 2: Pembrokeshire College
The evaluation at Pembrokeshire College also involved a group of {to be added} at {to be
added} and the digitised administrative system was carried out in parallel with the existing
paper based system and the outcomes compared.
Summary
{to be added}
Conclusions
{to be added}
Full Report
The full case study report can be viewed at: {to be added}
7

8. JISC PotsPan Project Final Report
Case Study 3: Swansea Metropolitan
The evaluation at Swansea Metropolitan was different to the other partners in that it explored
the use of electronic signatures for audit documents rather than digital pens.
Summary
The contribution to be made by Swansea Metropolitan University as a partner in the JISC
PotsPan project was to pilot the use of digital signatures with management and
administrative documents in the context of its merger with University of Wales Trinity Saint
David during the project period. The intention was to explore the benefits of electronic
signatures on documents that needed to be shared on multiple sites, particularly with regard
to the delivery of work-based learning.
The importance of this pilot exercise was that it was not constrained by the EU requirement
for hand written signatures that led to the use of digital pens for the other pilot exercises. It
was therefore an opportunity to explore the use and effectiveness of secure electronic
signatures on documents and how they could be used to authenticate digitally
communicated administrative paperwork.
The basic features of a secure electronic signature system are that it should be able to
confirm ownership of the document, that the signatory was authorised to sign the document
and that the document has not been changed since the signature was applied. Typically the
system will also identify the computer used to create the document and will verify the date
and time the signature was added.
A further element of security in the systems accepted by international finance and commerce
is the use of a third party Certification Authority (CA) that generates encrypted public and
private key certificates. The private key certificate is held on the owners’ computer and
ensures that the encrypted data includes verification information that is recognised by the
receiving computer(s) with the public key. If any aspect of the document and signature
security is not verified, then a ‘not valid’ alert will be shown.
Both commercial and open source document creation applications are available that include
the ability to add secure electronic signatures to documents and provide CA services. An
objective in the PotsPan project was to identify the most cost effective solution for the
institutions and this case study shows how open source solutions are both available and
effective.
The pilot exercise created and tested a series of documents with secure electronic
signatures. These documents were sent electronically as attachments and uploaded to
institutional websites and were shown to retain the encrypted data confirming signature and
document validity.
Conclusions
It was concluded that the work confirmed that electronic signatures that authenticated
digitally transmitted administrative documents were achievable in a cost effective way and
that met commercially accepted levels of verification and security. It is believed that
electronic signatures will eventually become acceptable for EU audit purposes and allow a
fully digitised online administrative system to be implemented.
Full Report
The full case study report can be viewed at: http://www.slideshare.net/ttoole/pots-pan-case-
study-swansea-met
8

9. JISC PotsPan Project Final Report
7. Outcomes
It is anticipated that the outcomes of the PotsPan project will have practical value, both for
the SAWSW WBL partnership, and for other groups looking for similar administrative
efficiencies using technology. However, there were certain compromises that needed to be
taken for the project to achieve its objectives.
The project took a pragmatic approach to the digitisation design, given the non-acceptance
by the EU of electronic signatures for audit purposes. Ideally, it would have prefered a fully
electronic signature system to be accepted, but, given the current rules, it adopted the digital
pen as a compromise solution.
This was not ideal as it still required paper versions of all the forms to be produced in order
for them to be physically signed. It did allow, however, the entire system to be digitised whilst
still satisfying EU requirements. This meant that the advantages of single data entry, data
consistency and the ability for management to access the system online were achieved.
This latter factor was perhaps one of the most important from a benefit point of view. The
paper based system was managed by Training Advisors and the documentation distributed
geographically as a result. This was both a risk and made central management access to
data very difficult.
However, the goal of achieving an acceptable electronic signature solution remains. The
outcomes of the PotsPan project are being disseminated with a view to contributing to and
furthering that debate. Presentations at Alt-C 2012 & the JISC Online Conference have
been part of that agenda.
8. Conclusions and Recommendations
The outcomes of the PotsPan project led the project team to conclude that the work had
confirmed the main benefits of administrative system digitisation:
Digitisation of the administrative system eliminated the need for the repeated entry of
core data and ensured data consistency;
Digitisation made the entire administrative system available to programme managers
at all times and from any location;
The system delivered significant operational efficiencies, mainly from the avoidance
of physically transporting learner files between different geographical locations.
However, the requirement for physical signatures on audit documents compromised the full
realisation of those benefits and the project team remained committed to advancing the
debate and argument for full electronic signature recognition for EU funded programmes.
The overall conclusion was that the PotsPan project had achieved its primary goal of
exploiting the benefits of the original Swani project and demonstrating its value and viability
in the practical administration and management of the work-based learning programme in
South West Wales. Furthermore, it was felt that the benefits were transferable to other
programmes with similar needs and hence contributed positively to the Jisc e-Learning
Programme Embedding Benefits objectives.
The recommendations arising from the project outcome included:
Noting that not only was a digitised administrative system beneficial from an
efficiency point of view, but also from a cost effective viewpoint: the document
management system used by the PotsPan being open source;
Noting also that there were open source electronic signature systems available that
met standard international security and authentication requirements;
Recommending that the current EU debate about the acceptability of electronic
signatures be concluded, not only for educational audit purposes, but for advancing
the wider free market goals of the union.
9

10. JISC PotsPan Project Final Report
10

11. JISC PotsPan Project Final Report
References and Appendices
All the project documents and additional materials are referred to as links in the text of this
report and/or are available on the project Wiki. For ease of access, the following links lead
directly to the key project documents.
1. PotsPan project Wiki: http://potspan.pbworks.com
2. PotsPan project Blog: http://potspan.blogspot.co.uk
3. Project Management:
http://potspan.pbworks.com/w/page/53357624/Project%20Management
4. The Digitised Administration System:
http://potspan.pbworks.com/w/page/53357697/Digitised%20WBL%20Administration
%20System
5. The Case Studies:
http://potspan.pbworks.com/w/page/53357795/Project%20Deliverables
11