Nationale Studenten Enquête

Nationale Studenten Enquête The organisation and practical use of the National Student Survey (NSE) in the Netherlands Paper presented in track 4 at the EAIR 35th Annual Forum in Rotterdam, the Netherlands 28-31 August 2013

Name of Author(s) A.V. (Lex) Sijtsma Drs. W.H.M. (Wil) Kwinten MSc T.C.C. (Theo) Nelissen

Contact Details A.V. Sijtsma Foundation Studiekeuze123 P.O. Box 2290 Utrecht, 3500 GG The Netherlands E-mail: [email protected]

Key words National survey, study choice, quality care, application of NSE

1

Nationale Studenten Enquête

Abstract Nationale Studenten Enquete The National Student Survey (NSE) is a nationwide survey among almost all students in higher education in the Netherlands. Students offer their opinion concerning study programme and institution. The NSE is a unique survey, not in the least by its collaborative approach. The results are used by a growing number of organisations. This paper elaborates on how the NSE is implemented, gives practical examples of its use and how it’s used within an institution of further education.

2

Nationale Studenten Enquête

Nationale Studenten Enquête The organisation and practical use of the National Student Survey (NSE) in the Netherlands

Table of Contents 1

Introduction ................................................................................................................................................................ 5

2

What is the NSE? ......................................................................................................................................................... 6 2.1

3

Authors and organisations................................................................................................................................. 6

Application of the NSE ................................................................................................................................................ 8 3.1

Choice of studies ............................................................................................................................................... 8

3.2

Quality ............................................................................................................................................................... 8

4

The questionnaire ..................................................................................................................................................... 10 4.1

5

Panel of Experts ............................................................................................................................................... 10

The process ............................................................................................................................................................... 11 5.1

Collecting information ..................................................................................................................................... 11

5.2

Field research .................................................................................................................................................. 12

6

Communications ....................................................................................................................................................... 13 6.1

Channels .......................................................................................................................................................... 13

6.2

Promotional materials ..................................................................................................................................... 16

6.3

Meetings .......................................................................................................................................................... 17

7

Results ....................................................................................................................................................................... 18 7.1

Publicly available benchmark file .................................................................................................................... 18

7.2

Institution file .................................................................................................................................................. 19

7.3

Fact sheet ........................................................................................................................................................ 19

7.4

Key file ............................................................................................................................................................. 19

7.5

NSE Online ....................................................................................................................................................... 19

7.6

Research justification ...................................................................................................................................... 20

8

Case study: Avans University of Applied Sciences .................................................................................................... 21 8.1

Vision ............................................................................................................................................................... 21

8.2

Quality management and management model .............................................................................................. 21

8.3

NSE input into Avans processes ....................................................................................................................... 22

8.4

Valuable information ....................................................................................................................................... 23

8.5

Sample project ................................................................................................................................................. 26

9

Conclusions ............................................................................................................................................................... 27

10

Appendices........................................................................................................................................................... 28

10.1

Invitation text .................................................................................................................................................. 28

10.2

Introducing Intomart GfK ................................................................................................................................ 29

10.3

Response development ................................................................................................................................... 29

10.4

Participating institutions NSE 2013 ................................................................................................................. 30

3

Nationale Studenten Enquête

Figures Figure 1 – Scores awarded by students .............................................................................................................................. 8 Figure 2 – Web site for institutions ................................................................................................................................... 13 Figure 3 – NSE Bijblijver .................................................................................................................................................... 14 Figure 4 – Registration of e‐mail questions ...................................................................................................................... 14 Figure 5 – Response report ............................................................................................................................................... 15 Figure 6 – Response report for full-time version of programmes by location .................................................................. 15 Figure 7 – Response by institution .................................................................................................................................... 16 Figure 8 – NSE Banner ....................................................................................................................................................... 16 Figure 9 – Poster ............................................................................................................................................................... 16 Figure 10 – 2013 benchmark file ....................................................................................................................................... 18 Figure 11 – Fact sheet for Avans, 2013 (detail view) ........................................................................................................ 19 Figure 12 – NSE Online ...................................................................................................................................................... 20 Figure 13 – PDCA cycle ...................................................................................................................................................... 22 Figure 14 – Scheduling Infographic ................................................................................................................................... 25 Figure 15 – Daily development of response ..................................................................................................................... 30

Tables Table 1 – 2012 colour board (detail view) ........................................................................................................................ 24 Table 2 – 2013 NSE participants ........................................................................................................................................ 32

All hyperlinks were checked 17 July 2013.

4

Nationale Studenten Enquête

1 Introduction This paper introduces the largest Dutch survey to measure student satisfaction with higher education: the Dutch National Student Survey (Nationale Studenten Enquête). Not only the organisation of the survey is discussed, but also its participants, quality management, the end results produced and the aspects that amplify its practical use and value. In the context of one institute of higher education, the specific ways in which the NSE is applied and leveraged are demonstrated. This paper offers stakeholders the tools to better leverage student satisfaction data and help improve the quality of the educational system.

5

Nationale Studenten Enquête

2 What is the NSE? The National Student Survey (NSE: Nationale Studenten Enquête) is a nationwide survey researching students’ satisfaction with the course programme they pursue at their university or university of applied sciences. Students are asked to assess both their institution and their course programme. Up until 2010, educational institutions each measured the satisfaction of their students via internal surveys, in addition to which a nationwide survey was conducted. Since 2010 these have been united into a single survey: the NSE. Almost all publicly funded institutions participate, as well as a large number of privately funded institutions. Over 72 institutions participated in the 2013 survey and more than 264,000 students (a response of 39.3%) completed the questionnaire – the highest figures to date. This makes the NSE a prime example of effective collaboration between Dutch institutes of higher education. A unique and valuable aspect of the NSE is that its results can be used for benchmarking: institutions and course programme committees can compare their results to those of their peers. Since 2010, the questionnaire has remained largely the same, allowing for the analysis of trends. In addition, parties outside educational institutions, such as commercial publishing houses and the Dutch government, make use of NSE results for ranking institutions and course programmes and setting performance goals.

2.1 Authors and organisations This paper was written by:   

A.V. (Lex) Sijtsma, Studiekeuze123, [email protected]. Lex Sijtsma is the NSE programme manager and supervises its annual execution. Drs. W.H.M. (Wil) Kwinten, Avans Hogeschool/University of Applied Sciences, [email protected]. Wil Kwinten is senior advisor for research, quality management and education development at Avans. T.C.C. (Theo) Nelissen MSc, Avans Hogeschool/University of Applied Sciences, [email protected]. Theo Nelissen, researcher at Avans, performs analyses on NSE data specifically relevant to Avans’ purposes.

2.1.1 Studiekeuze123 Stichting (foundation) Studiekeuze123 provides independent and reliable information on higher education study programmes and institutions without being influenced by commercial interests. The Board consists of delegates from the representative bodies of the higher education institutions: Vereniging Hogescholen (Higher Education Council), VSNU (University Association), and NRTO (Dutch Council of Training and Education), as well as the student organisations LSVb and ISO. Studiekeuze123 is financed by the Ministry of Education, Culture and Science (OCW) and implemented by SURF, the higher education and research partnership for ICT in the Netherlands. Studiekeuze123 carries out the annual NSE on behalf of the Ministry of Education, Culture and Science and in collaboration with institutes for higher education.

2.1.2 Avans Hogeschool Avans Hogeschool (University of Applied Sciences) serves 26,000 college students on the level of higher vocational education and employs approximately 2,200 employees. Avans University provides higher education in eight sectors and at seven locations in Breda, 's‐Hertogenbosch and Tilburg. Education is organised in 19 schools and is offered in about 55 different substantive course 6

Nationale Studenten Enquête

programmes (to the levels of associate degree, bachelor and master) in the form of full‐time, part‐ time and work‐study programmes. For several years in a row, it is chosen by national rankings, as the best large-scale Dutch university of applied sciences, with the most study programs in top positions and above-average student satisfaction.

7

Nationale Studenten Enquête

3 Application of the NSE The NSE serves two important goals in the field of education: 1. Informing students selecting a course programme by collecting the opinions of enrolled students, benefiting prospective students’ ability to imagine the nature and quality of the course programmes offered; 2. Improving the quality of course programmes offered by institutes of higher education. Below, each goal will be explained further.

3.1 Choice of studies 30 to 40% of Dutch students will in the first year of their university career switch to another course programme or drop out altogether: a worryingly large number, which is recognised as a societal issue. The NSE offers an instrument to strengthen the basis on which students choose a course programme and strives to aid students in making these choices. On the Web site www.studiekeuze123.nl objective and independent data is collected on almost all of the course programmes available in Dutch higher education. For each course programme, the average score awarded by students currently enrolled in the course programme is listed. Figure 1 shows an example of the Business & IT Management programme at Avans University of Applied Sciences. Scores are plotted along a Likert scale from 1 to 5, illustrated below by the “Acquired general skills” scores of 3.9 and 3.6 (for full‐ time and part-time respectively), where 5 stands for “very satisfied”.

Figure 1 – Scores awarded by students1

3.2 Quality Another goal of the NSE is to help advance quality management within institutes for higher education. Many institutions use NSE data alongside their own research to spot issues brought out by conspicuously low or high scores on certain themes. Their own internal research then enables them to investigate any issues in greater detail. One university of applied sciences, for example,2 scored low on time taken to score tests, as well as 8

Nationale Studenten Enquête

other items of contact and information (such as accessibility of the lecturers) and the feasibility of deadlines. Another university of applied sciences scored significantly higher. What could have caused this discrepancy? Further research and discussion brought to light a connection between the two scores, as there appears to be a link between students’ academic self‐ discipline and teachers’ professional selfdiscipline. The lesson herein was that these low scores should not only be attributed to students succumbing to procrastination, but also to the behaviour of teachers. The higher scoring university of applied sciences turned out to have a clear-cut “a deal is a deal” policy: If it was agreed that tests were scored and results posted within 10 working days, teachers who failed to perform were held accountable by their superiors as well as their peers. The lowerscoring institution turned out to lack such a policy. Agreements were seldom met and often adjusted; test scoring times were not standardised. As a results, students often found themselves in the dark. This serves as an example of general trends made visible through NSE data and issues that institutions can tackle with additional research as a basis for effective measures.

9

Nationale Studenten Enquête

4 The questionnaire The questionnaire was put together in collaboration with institutes of higher education. In its current form, the questionnaire is largely the same as it was in 2010; however, it was modified slightly in 2012 to comply with requests from the institutions. The questionnaire comprises about 100 questions, divided into the following categories:  Background information  General assessment  The content and organisation of teaching  Acquired general skills  Acquired scientific skills/skills for applied research  Lecturers in your study programme  Academic guidance/counselling  Testing and assessment  Information provided by your study programme  Programme schedules  Study load  Contact hours  Group size  Internships  Study facilities  Quality care  Other facilities and study environment  Living arrangements  Studying with a handicap  International students  The Ultimate Question  Wishes/ideas for improvement The questionnaire is available at http://www.studiekeuzeinformatie.nl/nse/over-nse/vragenlijst in English, Dutch and German. Not all questions are relevant to every student, which is why participants are routed to see only the questions that pertain to their situation. Dutch students, for instance, see no questions from the “International Students” section.

4.1 Panel of Experts The survey’s quality – in terms of representativeness, validity, reliability and applicability – is assessed by a panel of experts for Studiekeuze123, ensuring scientifically sound research. In addition, Studiekeuze123 employs a Data Quality Officer.

10

Nationale Studenten Enquête

5 The process The process of conducting the NSE survey starts September and ends in May. It comprises, roughly, three stages:  Aggregation of data (September – December)  Field research (January – March)  Delivery of results (March – mid‐May) Studiekeuze123 supervises the NSE; that is to say, Studiekeuze123’s role is to oversee and direct the execution of the survey. The operational realisation of the survey is in the hands of the specialised research institute Intomart GfK, based in Hilversum, the Netherlands.

5.1 Collecting information A first step is the collection of information sent in by participating institutions. The NSE is kicked off with an introductory meeting, so as to explain the procedure and the time schedule for the coming year. Studiekeuze123 sends all publicly and privately funded institutions in the Netherlands a written invitation to participate. This includes a contract to be signed and returned to Studiekeuze123. Following this, Intomart GfK contacts participating institutions asking them to send in the following data: 1. Student data. Institutions are asked to send in specific information on their students, including student ID number, e‐mail address, age, course programme, course year etc. This information is used to invite students to participate and is given back to the institutions after the survey. Institutions are to submit information about all of their students: selecting certain groups or individuals is not allowed. Certain categories are left out, however, such as students enrolled in a pre‐master programme. 2. Extra questions (optional). Participating institutions may add a limited number of their own questions, pertaining for instance to specific local circumstances. This way they can forego organising their own research. These questions can also aid in encouraging students to complete the questionnaire. The extra questions are added to the questionnaire and posed to students after the general questions. Answers are made available to the institution only. Intomart GfK can assist institutions in formulating their own questions and performs a number of standardised checks to ensure, for instance, that students’ privacy is safeguarded. 3. Extra labels (optional). To aid in the processing of the resulting data, institutions may add custom labels to the questionnaire; for example, in the case of course programmes that are the result of a collaborative effort by two institutions. In this case, the name of the collaboration can be added as a custom label. Labels are visible to the institution only. 4. Invitation text (optional). (See also Appendix 10.1.) Institutions or course programme committees may add their own afterword, as well as an additional line of text to be added to the standardised invitation sent on behalf of Studiekeuze123. These texts may even be student‐specific. Intomart GfK makes a secure uploading environment available to institutions. During uploading, some preliminary, automated checks are done: course programme names and institutions, for instance, are checked against public registries (BRIN and CROHO respectively). In addition, Intomart GfK carries out their own checks on these data, such as checks for missing course programmes. The data are subsequently added into the database on which the field research will be performed.

11

Nationale Studenten Enquête

5.2 Field research The field research phase takes 8 weeks from 2014 on, before which it took a total of 10 weeks, and starts out in mid-January. The NSE survey is offered online at www.nse.nl. For the 2014 NSE, research is underway into how the questionnaire can be comfortably filled in on a smartphone. Expectations are that this will increase response figures owing to the greater ease of use. All students are invited via e-mail. For every student, institutions may enter up to two e-mail addresses; commonly, a university address and a personal address. All e-mail addresses that are sent in are used, which has led to the impression that the use of personal e-mail addresses yields higher response figures and is thus more effective than using a student’s institutional address. Students may complete the questionnaire in instalments: the system will remember what parts are yet to be completed. In like manner, it is possible to switch from a PC to a laptop or tablet mid-use. Until students have completed the entire questionnaire, they receive a reminder every fortnight. In the last week of this phase, a so-called “last shot” reminder is sent. Institutions may track development of the response via response rate summaries. For more information on this, refer to section 6.1.7.

12

Nationale Studenten Enquête

6 Communications Adequate communications around the process of the NSE survey are of major importance. Studiekeuze123 keeps in touch with their contacts at institutions and with participating students via various channels of communication. Studiekeuze123 also supplies participants with promotional materials and organises a number of events around the NSE.

6.1 Channels 6.1.1 Web site www.nse.nl The survey is offered online at www.nse.nl.

6.1.2 Web site www.studiekeuzeinformatie.nl/NSE A Web site is maintained for participating institutions, offering information on the survey as it is underway: for instance, background information on quality assurance and research justification. Studiekeuze123 is firmly committed to keeping the entire NSE process open and transparent.

Figure 2 – Web site for institutions

6.1.3 Social media Studiekeuze123 regularly publishes tweets (hashtag: #nse2013) and enables students to send a tweet or publish a Facebook post to say they have completed the NSE survey.

6.1.4 NSE Bijblijver NSE Bijblijver is a digital opt-in newsletter that is sent out as needed over the course of the research phase. In 2013, 8 of these newsletters were sent out. An archive of past newsletters is available via http://www.studiekeuzeinformatie.nl/nieuwsbrief/archief-nieuwsbrief-en-bijblijvers.

13

Nationale Studenten Enquête

Figure 3 – NSE Bijblijver

6.1.5 E‐mail: [email protected] and [email protected] Participating institutions as well as students can get in touch with the NSE organisers via e-mail. The address [email protected] is available for institutions and students can use the address [email protected]. All incoming e-mail is automatically given an identification number and entered into a call handling system. This system is developed for the handling of questions and incidents and can be manned by multiple employees if necessary. As a call is handled, a record of all actions is kept. This allows for quick and adequate reactions to questions and ensures no questions remain unanswered, even in busy periods. About 2,000 e-mails were handled in 2013.

Figure 4 – Registration of e‐mail questions

14

Nationale Studenten Enquête

6.1.6 Telephone number: +31 35 625 76 68 The Intomart GfK team can be contacted via +31 35 625 76 68.

6.1.7 Response rate Participating institutions may track development of the response in real time. As soon as a student finishes the survey, the questionnaire is added to the response rates. Response rate summaries are available to institutions both in overview form and in several more detailed views. Please see Figure 5 for a look at the response reports available.

Figure 5 – Response report

The example below shows one of the views available for Hogeschool Avans, which offers full-time, part-time and work-study programmes. In this example, only the figures for the full-time versions of several course programmes are visible, split by location.

Figure 6 – Response report for full-time version of programmes by location

There are also overviews for institutions who wish to compare their own response rates to those of others. See below for an example of an institution-level overview.

15

Nationale Studenten Enquête

Figure 7 – Response by institution

Finally, Studiekeuze123 informs all stakeholders on the development of response rates on a weekly basis.

6.2 Promotional materials Studiekeuze123 supplies participants with promotional materials in the form of banners, flyers and a poster. The building blocks for these materials are also made available, so that institutions can add their name or other information. The materials are available in English, Dutch and German. Please refer to http://www.studiekeuzeinformatie.nl/nse/traject-van-nse/promotiemateriaal for these materials and more information.

Figure 8 – NSE Banner

Figure 9 – Poster 16

Nationale Studenten Enquête

6.3 Meetings Studiekeuze123 funds and organises several events for and in collaboration with participating institutions: to inform them on NSE, but also for the benefit of networking. The following meetings are organised: 1. Introductory and closing meetings. All institutions are invited to these two events, which are held at the start and the end of the survey period. 2. Knowledge Sharing Workgroup. Participating institutions can benefit greatly from sharing information, such as best practices, tips and tricks, and have shown great interest in learning from one another. To this end, a knowledge sharing workgroup was established among institutions. Each year, three meetings are organised to present and discuss a variety of topics. In 2013, the workgroup discussed how institutions can spread the results across their organisations; also, the commercially oriented party CHOI (www.choi.nl) explained how they make use of NSE data in producing their Keuzegids (Course Programme Guide). In addition, the workgroup sets up part of the agenda for the introductory and closing meetings. Three meetings were held in 2013. 3. Focus group. Institutions may share their opinions on future plans for the NSE through this focus group, which Studiekeuze123 assembles from a selection of participants. These institutions are asked about their experiences with recent developments within the NSE and a full evaluation of their NSE participation is performed every year. The focus group meets as needed, convening twice in 2013.

17

Nationale Studenten Enquête

7 Results When the field research phase is complete, the results are ready to be processed into the final product. The raw data are cleaned and pre-edited: incomplete questionnaires are removed, checks are performed and weighting factors are calculated and applied.

7.1 Publicly available benchmark file This SPSS file holds all answers given by all students; however, the data is stripped of background information such as age, gender, course year and so on. These results are synchronised with results of past years, to allow for trend analyses. Currently the file contains the results from 2010 to 2013, amounting to 900,000 records. Institutions can use it to compare their results to those of their peers.

Figure 10 – 2013 benchmark file

7.1.1 Licensing to external parties The public benchmark file is also licensed to parties outside participating institutions, who may use it to develop their own products. Four examples: 1. Elsevier magazine (www.elsevier.nl) annually publishes a research special entitles “University course programmes: Best of the Best”, a comparison of higher education programmes based in part on NSE data and professors’ assessments. 2. CHOI (www.choi.nl) produces the annual Keuzegids (Course Programme Guide). Aimed at pupils as well as students, this is a critical comparison of all course programmes available, sorted by field. 3. Studie in Cijfers developed a “quick facts” leaflet format for easy reference to objective figures and information on a course programme. Examples of these leaflets and more information can be found at www.studiekeuze123.nl. 4. Studiekeuze123 (www.studiekeuze123.nl) not only organises the NSE itself, but also leverages the resulting data. Studiekeuze123 supports prospective students in choosing a course programme by providing them with independent and reliable information on higher education study programmes and institutions without being influenced by commercial interests. The public benchmark file is also licensed to organisations working in the field of education, who use NSE data mainly for policy development. The Dutch Ministry of Education, Culture and Science uses NSE data to set performance goals for institutes of higher education. 18

Nationale Studenten Enquête

7.2 Institution file This SPSS file comprises the results for one institution only. It is an extract from the public benchmark file, enriched with background data and the institution’s own additional questions and labels.

7.3 Fact sheet A fact sheet lists an institution’s NSE results on a single page. The year’s scores are separated by theme and question and listed alongside the previous year’s scores to highlight any fluctuations. Studiekeuze123 also sends these fact sheets to participating institutions’ Boards of Directors.

Figure 11 – Fact sheet for Avans, 2013 (detail view)

7.4 Key file The practical use of NSE results is multiplied when the data can be linked to the institution’s internal administration system. The key file makes this possible. Results can also be linked up with the institution’s own research data and NSE results can even be analysed according to source group: for instance, underperforming students who went to the same secondary school, or students coming from another school and lacking mathematical skills. In order to safeguard privacy, Studiekeuze123 subjects the use of the key file to stricter security regulations.

7.5 NSE Online NSE results are made available in SPSS format, the use of which can take some training. This is why Studiekeuze123 also offers a simpler reporting tool: NSE Online, which is available at www.nse.nl/online.

19

Nationale Studenten Enquête

Figure 12 – NSE Online

7.6 Research justification The research institute responsible for the NSE’s operational realisation publishes a yearly research justification detailing how the survey is carried out. This in turn is assessed by the Panel of Experts at Studiekeuze123. The research justification is available at http://www.studiekeuzeinformatie.nl/nse/resultatennse/evaluatie-en-onderzoeksverantwoording. The 2013 version is set to be published in October 2013.

20

Nationale Studenten Enquête

8 Case study: Avans University of Applied Sciences This chapter offers a look at the value of student satisfaction figures to the improvement of education quality. As a case study, the approach of Avans University of Applied Sciences is presented. This section was in part inspired by NVAO’s description of Avans’ institutional quality system.

8.1 Vision Avans’ vision centres around collaboration, viewing education as something to be built together. This helps the institution ensure their vision is recognised and supported. Avans organises course programmes focused on the small scale and seeks to approach students in a personal manner. In spite of its size (26,000 students are enrolled), the institution has succeeded in making students feel they are individually recognised. In addition to the focus on small-scale design of its course programmes, Avans values personal interaction and collective decision making. The institution strives to incorporate these values in all departments of the University: not only in its formal structures and protocols, but also in its culture and atmosphere, harmonising with the individual departments’ discourse on their views on teaching at Avans.

8.2 Quality management and management model The quality management system in place at Avans, illustrated below, was developed based on Deming’s PDCA cycle (see also http://en.wikipedia.org/wiki/PDCA). This is a management method consisting of four steps: PLAN, DO, CHECK and ACT. Avans utilises this method on three levels of its organisation: 1. University level (divided into academies, teaching locations, etc.) 2. Academy level (divided into course programmes) 3. Course programme level (divided into full‐time/part‐time/work‐study and teaching locations) The individual cycles are interconnected and information is fed into them from formal as well as informal circuits. Types of information input to these cycles include recurring evaluations (e.g. periodic education evaluations; the NSE; the HBO Monitor graduate survey; employee satisfaction surveys; the NVAO Institutional Audit), as well as recruitment and transfer figures. This is the path student satisfaction data follow into the organisational levels of Avans. This process is explained in more detail below.

21

Nationale Studenten Enquête

Figure 13 – PDCA cycle

Quality care and quality management can be approached from a variety of angles. Avans is invested in going beyond just the management of quality and creating a culture of quality. In his article, 3 Dries Berings paraphrases Peters and Waterman: “Mastering the soft side of managing (Shared Values, Skills, Style, Staff) is just as important as mastering the hard side (Strategy, Structure, Systems).” This can be taken to mean that the formal systems need to lead into the informal systems: that is, into the culture of the organisation. Organisational culture, as Berings quotes Houtveen (1996), can be defined as “the often unaware and difficult to verbalise set of common values regarding the work in its broadest sense, as shared by the members of the organisation” (translated on behalf of authors). The focus on a culture of quality is an integral part of the Avans management model, which was based on Robert Simons’ four Levers of Control.4 This model revolves around being in control without hindering creativity or employees’ own sense of initiative. The management model addresses both the formal and the informal aspects of management.

8.3 NSE input into Avans processes 8.3.1 Examples of NSE input and monitoring into formal Avans processes – Avans is implementing a data warehouse: an integrated information system to unite planning and monitoring data in a “dashboard” overview. NSE results will be available for viewing, as well as other key performance indicators of critical business processes. One such performance indicator is the fact that 75% of students award their general satisfaction a score of 4 or 5 – one of the performance goals set for Avans by the Ministry of Education. – At the start of each academic year, academies present their business plan containing the goals they aim to realise in the coming year. The basis of these goals is formed in parts by NSE results. These plans are also discussed with the Board of Governors, as are focal points arising from the NSE results. In the course of the year, academies share their progress via quarterly management reports.

8.3.2 Examples of NSE input and monitoring into informal Avans processes – The scores Avans is awarded, on an institutional level as well as per academy or course programme, are discussed throughout the institution. Positive results are celebrated and broadcast 22

Nationale Studenten Enquête

via posters, prints on coffee mugs, cakes, articles in the school paper, and so forth. In brief, the NSE is very much a topic of discussion. Attention is also paid to communicating positive results to prospective students who have yet to choose a course programme. – Internal rankings result in healthy competition between course programmes and academies. In this, Avans attaches great importance to full disclosure of NSE results. This policy of complete transparency is central to Avans culture and the institution considers the “naming and shaming” aspect vital to the high scores Avans has attained in the past four years.

8.4 Valuable information Formal processes and systems only play an indirect role in the realisation of quality. They are a means to an end: that of actual changes in behaviour. In this context, the way information is made available is important: Data should be presented in an accessible format, from which meaning can be deduced easily from the perspective of those involved. It should be noted that many improvements should ultimately be realised on the lowest level. After all, students are working on their course programme, not on the academy or the university. Information on higher levels in the hierarchy are more likely to lead to provisional or managementrelated measures. Conversely, real improvement is made in the primary process – among lecturers, students, in learning activities and so forth. The consequence of this view is that Avans results need to be primarily communicated to lecturers, committees (such as programme committees and the examination board) and directly to the students themselves. The NSE results allow for this type of communication; for instance, the lecturers can get feedback on how their teaching efforts on a specific course programme are assessed. Especially where the softer aspects of management are concerned, NSE results can be effectively leveraged owing to the support the NSE data generates. Stakeholders within Avans trust the information given, meaning they are more inclined to put it to use.

8.4.1 Ways of sharing information – When NSE results become available, each academy, as well as the Board of Governors, receive an overview of the results that are most important to their department. These “colour boards” are colour-coded for improved readability: progress and decline are immediately visible. These overviews are printed on A3/Ledger-sized paper, laminated and given to the management and the Board of Governors. Even though the results are available in digital form, this highly readable presentation of results proves considerably more effective. The colour boards are pinned to the walls of nearly every management office.

23

Nationale Studenten Enquête

Question Your study programme in general The content of the study programme The general skills acquired in your study programme Applied research within your study programme The preparation for a professional career The lecturers in your study programme The information provided by your study programme The study facilities offered by your study programme Testing and assessment Programme schedules Study load Academic guidance/counselling Other facilities and study environment The general atmosphere in your study programme The degree to which you are involved in improving your study programme Would you recommend your programme to friends, family or colleagues?

Total for Avans

Academy 1

Academy 2

Academy 3

3.89

3.90

3.67

……

3.85

3.93

3.68

……

3.77

3.78

3.57

……

3.67

3.64

3.38

……

3.56

3.54

3.34

…..

3.71

3.63

3.28

…..

3.48

3.53

3.18

…..

3.62

3.80

3.20

…..

3.52 3.45 3.49

3.56 3.22 3.50

3.26 3.38 3.33

….. ….. …..

3.53

3.57

3.24

…..

3.62

3.74

3.34

…..

4.16

4.10

4.06

…..

3.64

3.62

3.40

…..

4.17

3.99

3.97

…..

Table 1 – 2012 colour board (detail view) – Directors, academy employees and students all receive an academy report. This report presents the scores specific course programmes were awarded.

– NSE results from 2010 onwards are available via a Qlikview application. This convenient application offers the possibility to freely explore the data and to generate ready-made reports answering frequently asked questions. As part of the institution-wide Management Information Programme, access to NSE results will be offered via Business Objects 4.0 from 2014 on. – For simple questions, a central help desk is available to generate basic reports. – Following requests from the Board of Governors as well as academy directors, a comprehensive analysis of the NSE results can be performed as needed. – NSE Specials are published about salient issues, in which relevant information from the NSE is analysed and provisional conclusions are drawn. These Specials can be enriched with infographics and supported with other sources of information. For a sample Special on scheduling issues, see 24

Nationale Studenten Enquête

below.

Figure 14 – Scheduling Infographic

25

Nationale Studenten Enquête

8.5 Sample project The scores awarded on the subject of programme scheduling resulted in a degree of internal debate on their accurate interpretation. NSE results showed a score of 3.45 on Scheduling, one of the lowest scores across the themes. Although low, this score is adequate: on the scale used, a score of 3 counts as “neutral” and any higher scores signify satisfaction. Further research within Avans revealed room for improvement in the internal scheduling process. Students turned out to be bothered mainly by the late publication of programme schedules and the frequent changes to the schedule. The Board of Governors referred to this information in a request to both the support services and the academies to improve the situation. To accomplish this, the following goals were set and linked to NSE results:  The quality of programme schedules is rated as “good” by students, meaning a minimum average NSE score of 3.65 (2011: 3.45) and no academy is rated below 3.3.  The programme schedules are published in time, meaning a minimum average NSE score of 3.25 (2011: 3.05) and no academy is rated below 3.0.  Notifications of changes to the current schedule are clearly communicated, meaning a minimum average NSE score of 3.22 (2011: 3.02) and no academy is rated below 3.0. A project has been launched to achieve this. The goals of this project are as follows:  The timely publication of the programme schedules  Better schedule quality  A more flexible approach to scheduling  A less error-prone approach to scheduling  Fewer changes to the schedule  Better communication of schedule changes

26

Nationale Studenten Enquête

9 Conclusions This paper introduced the foundations of the Dutch National Student Survey and ways in which the results may be leveraged within an institution. Examples were given of the application of NSE data and of survey questions, realisation of the research, communications and licensing of the results. A case study was presented in the form of Hogeschool Avans’ application of NSE results: both in general and illustrated by the programme scheduling project. It is hoped that this paper presents the reader with ample information to assess the benefits of NSE for institutions of higher education. The authors and the NSE staff will be glad to answer any questions via [email protected].

27

Nationale Studenten Enquête

10 Appendices 10.1 Invitation text The invitation and all the reminder e-mails are available in Dutch, English and German. This is the template for the English text. Words/sentences between [] and are metaterms that are replaced with the actual content.

NSE 2013 (29535) Uitnodigingsmail Versie 3 Subject: Give your opinion about your studies at [naam instelling]. Sender: National Student Survey 2013 English Dear student, Who is able to rate your study programme and [hogeschool/universiteit] better than you? This is why we’re inviting you to participate in the National Student Survey 2013, the leading annual survey into the quality of programmes in higher education. The National Student Survey: why?  Your feedback will help prospective students to make the right choice when it comes to a study programme. The results of the survey will be published on the independent comparison site Studiekeuze123.nl.  Your feedback will help [hogeschool/universiteit] to improve the quality of their education. Take the survey [Your e-mail address has been provided by your institution, which will benefit from your participation.] [Of in plaats van de vorige zin de instellingstekst.] The more students who participate, the better the results will be. It’s easy to take the survey: just click on the link below and you can start straight away. [persoonlijke link naar de vragenlijst] If this link does not work, then copy the link into your browser’s address bar.

Completing the survey will take no more than 10 to 20 minutes. You can fill in part of the survey now, and return to complete it later if you like. Just click on the link again, and you will automatically pick up where you left off. Win a great prize! We’ll be giving away fantastic prizes to some lucky participants, including:  5 iPhone 5s (value: €800).  5 iPad minis (value €400).  240 Bol.com vouchers (value €25).

28

Nationale Studenten Enquête

Studiekeuze123 Foundation The National Student Survey is the responsibility of the Studiekeuze123 Foundation. The foundation is an initiative of the student organizations (ISO, LSVb) and the educational umbrella organizations (HBO council, VSNU and NRTO). The research firm Intomart GfK is conducting the survey on behalf of Studiekeuze123. The results of the survey will be available from mid-June at Studiekeuze123.nl. If you are having problems logging in or if you have any questions about the survey, please send an email to [email protected] or see www.nse.nl/faq for more information. Thank you very much for your participation! Kind regards, Tom Dousma Studiekeuze123



Please note: intends to use the results of the survey for further statistical analysis. These analyses will be used to enhance the quality of education and improve the institution as a whole. The institution needs the background information on the students who participated in the survey in order to complete these analyses. The relevant data will only be used by the personnel responsible for Institutional Research at , and will never be provided to third parties. The data will be treated with the utmost confidentiality. The survey results will be anonymized and cannot be traced back to individual persons. The reports will also be completely anonymous. Click to unsubscribe to the National Student Survey 2013.

10.2 Introducing Intomart GfK Intomart GfK describes itself as follows: GfK is one of the world’s largest research companies with a primary focus on realizing ‘growth from knowledge’ for its clients. GfK Intomart, as part of the GfK concern, focusses on three main areas: Media, Marketing and Public Services. Within Public Services, GfK Intomart works for a broad range of non-profit clients, also addressing a broad range of research subjects. Data collection is normally (still) an important aspect of our services, but the next step, fact-based consultancy, nowadays becomes more and more important since our clients express the need to use our expertise in interpreting and implementing the outcome of the research we conduct. Website: http://www.intomartgfk.nl/

10.3 Response development The figure below shows the daily development of the response to the 2013 NSE. These data represent both usable and unusable responses and are recorded before any cleaning or checks are performed. If only usable responses are considered, totals are about 1% lower. 29

Nationale Studenten Enquête

Figure 15 – Daily development of response

The peak in student responses on 21 January coincides with the sending of invitations to the students’ personal e-mail addresses.

10.4 Participating institutions NSE 2013 No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 30

Institution ACTA Amsterdam University College Amsterdamse Hogeschool voor de Kunsten ArtEZ hogeschool voor de kunsten Avans Hogeschool CAH Vilentum Dronten / Almere Christelijke Hogeschool Ede Christelijke Hogeschool Windesheim De Haagse Hogeschool De Kempel Design Academy Eindhoven Driestar educatief Erasmus Universiteit Rotterdam Fontys Hogescholen Gereformeerde Hogeschool Gereformeerde Hogeschool II Hanzehogeschool Groningen HAS Den Bosch Hogeschool Edith Stein/ Onderwijscentrum Twente

Nationale Studenten Enquête

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 31

Hogeschool Helicon (nu Hogeschool Leiden) Hogeschool INHOLLAND Hogeschool IPABO Amsterdam Alkmaar Hogeschool Leiden Hogeschool Notenboom Hogeschool Rotterdam Hogeschool Schoevers Hogeschool Tio Hogeschool Utrecht Hogeschool v Pedagogisch en Sociaal-Agogisch Onderwijs Groningen Hogeschool van Amsterdam Hogeschool van Arnhem en Nijmegen Hogeschool van Hall Larenstein Hogeschool voor de Kunsten Utrecht Hogeschool Wittenborg Hotelschool The Hague HZ University of Applied Sciences Iselinge Hogeschool IVA Driebergen Juridische Hogeschool Katholieke Pabo Zwolle Markus Verbeek Opleidingen Stichting Hoger Onderwijs Marnix Academie Marnix Academie/Interactum NHL Hogeschool NHTV NHTV/Wageningen University Nyenrode Business Universiteit Open Universiteit Nederland Penta Nova Politieacademie Protestantse Theologische Universiteit Radboud Universiteit Nijmegen Rijksuniversiteit Groningen Saxion Hogeschool Saxion Next Stenden Hogeschool Stichting Hoger Onderwijs NOVI Stichting Hogeschool van Amsterdam Stichting NTI Hogeschool Stoas Wageningen | Vilentum Hogeschool Technische Universiteit Delft Technische Universiteit Eindhoven The New School for Information Services Theologische Universiteit Kampen Theologische Universiteit v.d. Christelijke Gereformeerde Kerken

Nationale Studenten Enquête

66 67 68 69 70 71 72 73 74 75 76 77 78 79

Tio University TMO - Hogeschool voor Modemanagement transnationale Universiteit Limburg Universiteit Leiden Universiteit Maastricht Universiteit Twente Universiteit Utrecht Universiteit van Amsterdam Universiteit van Tilburg Universiteit voor Humanistiek Vrije Universiteit Amsterdam Wageningen University Windesheim Flevoland Zuyd Hogeschool Table 2 – 2013 NSE participants

1

Most screenshots are only available in the Dutch language. Many thanks to Frank Steenkamp (www.choi.nl) for this example. 3 Berings, D. (2010). Kwaliteitscultuur in het hoger onderwijs: de bijdrage van organisatiecultuur aan de ontwikkeling van kwaliteitszorg, Tijdschrift voor Hoger Onderwijs & Management, 4-10, p. 51-57. 4 Simons, R. (1995). Control in an age of empowerment. Harvard Business Review, March-April 1995, p. 80-88. 2

32

Nationale Studenten Enquête