Toolkit and Films

1. Qualifications, qualifications frameworks, qualification systems
2. A new understanding of qualifications
3. New versus old ways of getting organised
4. Competing ministerial agendas
5. Making qualifications frameworks work
6. Towards sustainable qualification systems
7. Conclusions and recommendations

1. Qualifications, qualifications frameworks, qualification systems
While getting organised is complex and requires careful thought and precision, it is also urgent business for everyone concerned. We cannot over-state the lost opportunities that will arise if systemic change is not initiated, nor the lost benefits of revitalised and relevant qualifications to millions of people. To take this further in any given country context means addressing what we have called the hardware, the critical infrastructure for organising an effective and efficient qualification system. In order to make effective system-wide and system-deep reform there needs to be a clear understanding of the distinction between the term ‘national qualifications framework’ and the qualification system as a whole. We propose the following definitions:

National qualifications frameworks (NQFs) are tools which classify qualifications according to a hierarchy of levels, typically in a grid structure. Each level is defined by a set of descriptors indicating the learning outcomes applicable at that level. Levels vary in number as determined by national need. Qualifications are allocated to NQF levels based on learning outcomes. An NQF helps thus to classify the qualifications in order to distinguish and to link them. NQFs can have additional functions in terms of criteria for describing qualifications (e.g. by type, purpose, pathways, unit structures, or credit values) and for adopting qualifications to the NQF register. An NQF brings order to the landscape of qualifications. A national qualifications framework is thus a specific policy instrument that functions as a tool within an overall qualification system.

A qualification system is everything in a country’s education and training system which leads to the issuing of a qualification; schools, authorities, stakeholder bodies, laws, institutions, quality assurance, and qualifications frameworks. All countries have qualifications, so all have qualification systems. Qualification systems are the set of organisational arrangements in a country that work together to ensure that individuals have access to, and can choose and obtain qualifications that are fit for purpose, meet the needs of society, and offer opportunities for employment, recognition, career development, and lifelong learning.

While every country has a qualification system, an NQF is a specific instrument within a qualification system, and therefore not all countries have them.

All partner countries which are reforming their qualifications towards outcomes-based qualifications are using an NQF as the principal tool to achieve this change. But NQFs do not always succeed in linking different types of qualifications. Even with a framework that is conceptualised and agreed by stakeholders, the different sectors within a country’s education and training system may apply different principles for learning outcomes, quality assurance, and qualification standards. Such an NQF will be fragmented and not based on the common principles that should be an integral part of a common framework.

Partner countries’ qualifications and qualification systems are at different stages of development. We distinguis five stages of development, from the ad hoc stage where discussions about qualification reform is taking place but there are not yet plans for a policy or implementation programme, until the consolidated stage where curricula, assessment and learning adapt to new qualifications and individuals use new qualifications for career progression and mobility. (See the Annex on policy stage indicators).
In this chapter, we look at new and old ways of getting organised and how to make national qualification systems work. But first we need to get a common understanding of what we mean with ‘new’ qualifications.

2. A new understanding of qualifications

According to the European Qualifications Framework (EQF), a qualification is “the formal outcome of an assessment and validation process which is obtained when a competent body determines that an individual has achieved learning outcomes to given standards.” For many countries, this remains more conceptual than real. A new understanding of qualifications is spreading into policy documents and laws, but is not yet common among stakeholders, let alone the general public. The aim here is to be consistent in our understanding of the term ‘qualifications’, and to encourage partner countries to adopt internationally compatible definitions.

Other technical terms make the concept of qualification even more complex. For example, full vs part, higher vs vocational, or formal vs non-formal qualifications. Definitions of the terms ‘knowledge’, ‘skills’, and ‘competences’ can be equally confusing, particularly when addressing competences. Are competences only a matter of autonomy and responsibility, or much wider? Do they include attributes and attitudes? And do they cover an individual’s potential, or just their proven abilities?

There are important cultural differences affecting how far one can go with general and basic competences, or the extent to which qualifications can be composed of units. Some countries emphasise the importance of mastering a profession or trade, insisting that a qualification cannot be split into pieces. For them, the whole is more than the sum of its parts. Others are more pragmatic, putting the emphasis on the skills and competences that can be used for different career purposes. In many partner countries, as in some central European countries, there has been a strong tradition of professionalisation associated with qualifications. Recent transitions in partner countries have seen a process of de-professionalisation in which young people try to postpone specialisation, staying in education longer to keep their options open. Attainment levels have gone up and people are generally better educated, with improved generic skills, but this has not led to better qualifications. On the contrary, trust in existing qualifications has declined because of factors such as the proliferation of courses and qualifications, and the perceived gap between provision and labour market needs. At the same time, there has been a rediscovery of qualifications as a central policy issue, with a renewed emphasis on relevance, quality assurance, assessment, and recognition.

Many countries are moving towards integrated lifelong learning systems, and away from separate and often unconnected pillars for general education, VET, higher education, and adult learning. A national qualifications framework is a strategic instrument for facilitating lifelong learning, but even more fundamentally, the qualifications themselves can be the starting point for transforming learning processes, expressed in learning outcomes, as the products of education and training systems. The movement towards new qualifications as the core of integrated lifelong learning systems can be shown as a continuum, because rates of change vary from country to country. However, modern qualifications are significantly different from their traditional counterparts, as shown in Table 1.

Qualifications comprise learning outcomes defined in terms of knowledge, skills, and competences, for example, which provide measurable indicators against which an individual’s capabilities can be assessed. Work-related competences in occupational standards facilitate the definition of learning outcomes, and many partner countries have embraced occupational standards as a basis for developing relevant vocational qualifications. A learning outcomes approach can make the results comparable, and at the same time offer learners different pathways to achieve these results. But more attention must be paid to assessment and quality assurance in order to check that intended learning outcomes have been actually achieved.

Table 1. Modern and traditional qualifications

For providers, this means moving away from a traditional, norm-referenced approach where student performances are compared to each other, towards testing specific learning outcomes in national standards. While this reduces their ability to award qualifications at their discretion, learning outcomes allow providers more freedom in defining learning processes. Recognising that learning outcomes can be acquired through different pathways also enables the development of systems for the validation of non-formal and informal learning. Learning outcomes can facilitate the comparison of qualifications, if they are coherently expressed, particularly for those qualifications in affiliated areas that can be allocated to an identical level in a qualifications framework. This makes it possible in principle to compare qualifications that are developed and awarded by different institutions.

The view that qualifications do not matter, and that what is important is having the skills to succeed, is still heard. But this is a simplification that needs to be challenged. Skills are important, especially in continuing vocational training, but for someone to show that they possess a set of skills demands some form of portable currency, i.e. a qualification. Good qualifications capture what knowledge, skills, and competences people need in order to be equipped for the future labour market. Such qualifications are a necessity when people increasingly move between jobs and between national labour markets. A new understanding of qualifications should also cover part qualifications or units (where a unit is a specific set of learning outcomes) to facilitate validation of non-formal and informal learning. Qualifications establish the all-important link between education and work, creating a common language among providers, learners, and employers.

The NQF concept, promoted through the EQF, has turned existing concepts of qualifications in partner countries on their head. Qualifications have always been seen as the logical outcome of a curriculum, the end result of the learning process. But as our previous study demonstrated, better results start from learning outcomes, and curricula need to be developed from qualifications, not the other way round. Another change is that qualifications are being used as formal certificates, while people commonly still refer to someone’s qualifications as their competences. These are deeply-rooted differences in the perception of qualifications, and they are only gradually changing.

3. New versus old ways of getting organised
In many partner countries the whole set of necessary arrangements to qualify learners can perhaps best be characterised as being in flux. There is innovation taking place, and there are new laws, strategies, and regulations being adopted that embrace modern concepts of qualification. There are pilot projects and experiments in developing occupational standards, qualifications, and curricula. But most vocational qualifications are not yet based on learning outcomes and remain weak on assessment, and they have not been developed with systematic input from the world of work.

Where stakeholders from the world of work have started to engage, and are cooperating in developing standards and qualifications, capacities and resources are inevitably limited. Some countries get stuck at the legislative level. And countries cannot advance on the basis of voluntary cooperation between stakeholders alone; they need systemic approaches, in both the software (concepts) and hardware (operational arrangements) of qualification systems. They also need to review existing qualifications and develop hundreds of new ones. They need to establish repositories in the form of databases that are available to users, along with methodologies, guidelines, rules and regulations, procedures, resources, and institutions – and to build capacities in all of these components.

This does not imply that vocational qualification systems in the partner countries have been completely without links to the labour market. On the contrary, many countries, particularly in the Balkans, Eastern Europe, and Central Asia, have inherited systems of vocational qualifications that are intertwined with labour market regulations. For instance, the tariff-based qualification system of the former Soviet Union regulated all permitted occupations and job titles. The Classifier of Occupations was linked to handbooks of qualification characteristics that described the skill requirements for each occupation. These qualification characteristics were, in turn, the basis for developing vocational education standards and professionally-oriented higher education standards. National lists of educational programmes or specialisations determined which state education standards had to be developed. The state education standards contained the requirements for the provision, as well as for certification. Because they regulated the requirements for certification, they could be called the qualification standards. The qualifications that were obtained regulated access to occupations and jobs, and were part of the formal labour registration system. The diplomas that were issued after completion of the studies mentioned both the area specialisation and the ‘occupation’ (kvalifikaciya in Russian) that was obtained by the holder. People were subsequently registered by their qualification/ occupation in their workbook, the job registration booklet that every worker had, along with the related wage level, which would normally increase with their responsibilities after performance assessment. Qualification and wage level also determined working conditions and pension arrangements. Similar arrangements existed in the former Yugoslavia.

More than 25 years after the collapse of the Soviet Union, much of this system survives in one form or another, particularly where there is still considerable wage employment. And even longer after widespread de-colonisation, elements of the former British and French education systems can still be traced in partner countries from the Southern and Eastern Mediterranean region.

4. Competing ministerial agendas
Qualifications are an important topic in both education and labour market policies. While education ministries have been focusing on curriculum reform, and in particular widening existing programmes, labour ministries have been trying to ensure that occupational descriptors reflect changing labour market needs. It is often the labour ministries that started to work with employers’ representatives or social partners on training programmes for job seekers and certificating adult learning. This must be seen against the background of growing unemployment and economic restructuring, requiring the development of better adult learning to support retraining and career change. These initial competency-based programmes and qualifications have also had some impact in curriculum reform in secondary vocational education, under the influence of donor projects. But although curricula have changed, qualifications have not always been affected. Qualifications are still defined by state educational standards, and remain the outcome of the same or similar development processes.

The new NQFs promote relevant, quality-assured, learning outcomes-based qualifications that can facilitate lifelong learning, career development, and labour mobility. But apart from regulated professions, qualifications are not generally seen as an instrument for labour market regulation. On the contrary, qualifications should be passports to a wide range of career, learning, and personal development opportunities. This is appropriate for people who are expected to change their job role more frequently, with traditional wage employment much less common.

The new NQFs promote relevant, quality-assured, learning outcomes-based qualifications that can facilitate lifelong learning, career development, and labour mobility. But apart from regulated professions, qualifications are not generally seen as an instrument for labour market regulation. On the contrary, qualifications should be passports to a wide range of career, learning, and personal development opportunities. This is appropriate for people who are expected to change their job role more frequently, with traditional wage employment much less common.

The NQF allows the attribution of levels to qualifications issued by different organisations. Based on their outcomes, qualifications can receive a level. The learning outcomes make it easier to compare different qualifications for the same occupational area or field of learning, issued by different institutions. Learning outcomes make it possible to compare the results of learning in different contexts. This challenges the monopoly of education ministries as providers and issuers of qualifications. Employers and labour ministries are particularly attracted to the idea of learning outcomes-based qualifications that are responsive to labour market needs. The debate is once more about qualifications and what you can do with them, rather than educational programmes.

In moving to a new concept of qualification systems, with NQFs at the core, many issues require clarification. For instance, if new qualification systems are developed to support lifelong learning, which qualifications should be part of these NQFs? How are qualifications managed and quality assured? How can different types of qualifications be linked? What should happen to existing or obsolete qualifications? Which parts of the old system can be continued, and which must change? Many of these questions can only be answered over time, when implementation is sufficiently advanced.

5. Making qualifications frameworks work
We see national qualifications frameworks as vital tools in the systemic reform of education and training systems in our partner countries. Moving from NQFs as a concept to functioning frameworks populated with qualifications is a first critical step. An NQF without qualifications in it will have no impact. But populating a NQF raises many issues such as: which qualifications are good enough to enter the NQF register, who can propose the qualifications for the register, who checks their quality and approves them and who manages the register? These are aspects of the wider qualification system, rather than the NQF itself. And these questions cannot be answered by one actor alone.

In the narrow sense, the NQF provides a skeleton of levels to which qualifications can be allocated. The NQF as a classification instrument is a tool for bringing order to the landscape of qualifications. This is an important function, and the NQF is becoming indispensable for modern qualification systems. It can facilitate the comparison of qualifications at national and even international level, and brings together everything in one organised structure. But to make qualification frameworks work we must address it as part of the wider qualification system, covering all the arrangements that affect how qualifications are designed and developed, how they are managed, and how they are used for learning, assessment, and recognition in the education system and the labour market.

The focus in this toolkit is more on the qualification systems, rather than on qualifications frameworks. Previous studies have not made that distinction very clear. The European Parliament and Council Recommendation which established the EQF makes the following distinction:

‘national qualifications system’ means all aspects of a Member State’s activity related to the recognition of learning and other mechanisms that link education and training to the labour market and civil society. This includes the development and implementation of institutional arrangements and processes relating to quality assurance, assessment and the award of qualifications. A national qualifications system may be composed of several subsystems and may include a national qualifications framework;

‘national qualifications framework’ means an instrument for the classification of qualifications according to a set of criteria for specified levels of learning achieved, which aims to integrate and coordinate national qualifications subsystems and improve the transparency, access, progression and quality of qualifications in relation to the labour market and civil society;

However, these definitions are problematic, as they try to be both comprehensive and brief. In the NQF definition, the classification function is clear, leading to an understanding of how it could be used to integrate and coordinate national subsystems. But this function is not fulfilled by the NQF alone. Instead, it requires the involvement of stakeholders and institutions. Can it really be claimed that NQFs improve “access, progression and quality of qualifications in relation to the labour market and civil society” if there is no involvement of actors in the system, or principles to guide the development and use of qualifications?

The first sentence of the qualification system definition, on the other hand, is so wide-ranging that it can include complete education systems. Meanwhile, the second sentence looks narrowly at the institutional arrangements and processes for quality assuring, assessing, and awarding qualifications. In the third sentence, subsystems might have been explained. Does this refer to qualification systems for general, vocational, higher education, and adult learning? Or is it sectoral or field specific subsystems, or systems falling under the responsibilities of different ministries and other entities?

Figure 1. Different scope and characteristics of NQFs.

Figure 2. National qualification system of Lithuania.

The International Labour Organization explored these concepts further, asserting that the NQF as a driver of outcomes-based qualification systems could undermine the focus on strong education institutions. They highlight how NQFs in some English-speaking countries helped to create “distinctive features” that tend to separate the qualifications from the institutions which deliver them. They point out that “the nature and design of the NQF should be based on the goals that policy makers seek to achieve by introducing an NQF”.

Evidence from our partner countries shows that the development of outcomes-based systems is accompanied by efforts to improve provision, and that implementing new qualifications without improving curricula, provision, and teacher training is a dead-end that cannot produce more effective systems. Building on this evidence from partner countries, and beyond, affords the opportunity to look at how real, rather than ideal, qualification systems will be organised when countries move from traditional to outcomes-based models. In that sense, a better distinction between the framework and the system is vital, while avoiding comprehensive definitions that include overlapping aspects which are difficult to disentangle.

Figure 1 shows scope and characteristics of NQFs, which vary from a classification tool based on levels to a comprehensive framework. The latter includes the design of qualifications, principles of how learning outcomes should be described, programme design, assessment, validation, and quality assurance, along with levels. This identifies ‘push’ and ‘pull’ factors (indicated by the arrows) in organising the qualification system, including the scope of the framework, coordination mechanisms, the degree of regulation, and the responsibilities of actors. As Figure 1 also suggests, the quality of qualifications is affected by how they are organised.

Figure 2 shows another schematic representation, depicting the NQF playing a key role at the heart of the Lithuanian qualification system. The NQF brings order to the design and acquisition of qualifications, and to assessment and recognition, thereby supporting management of the system as a whole. This model is organised around processes that are not system-specific, as it describes functions rather than mechanisms and actors.

6. Towards sustainable qualification systems that can produce better qualifications
Qualification systems are effective if the organisational arrangements work properly together to achieve the outcomes described above. This means creating systems of interdependencies that can generate high-value qualifications, the effects of which can only be measured when individuals have actually been certificated. However, improvements to the way systems are organised and structured can be made now. We have identified four foundation components in the organisation of a qualification system, which are common to all systems and are independent of local or other specific environmental factors.

These are:
1. The legal and regulatory context
2. Effective stakeholder dialogue
3. Institutional arrangements, and
4. Quality assurance arrangements.

Rather than looking at national differences we wish to identify the commonalities for successful systems. Within these four fundamental building blocks, then, we are looking for the most effective formula or set of arrangements.

Laws or regulations stipulate functions of the NQF and criteria for qualifications, and allocate tasks and responsibilities to associated institutions. They also regulate the rules of the game so that each party can play their role fully within the system. Laws or regulations often specify the practical purpose of the NQF, articulating the basic requirements for qualifications that are part of the framework, their relationships, and how they are used. Legislation is needed to enable reform and confirm changes in policies, and to regulate the qualification system. This helps to facilitate the quality and comparability of individual qualifications, and ensures the necessary resources and capacities are set aside to move from pilots to system-wide implementation. Laws can be enablers, but can also create rigidities that only inhibit reform. Legislation is a process, and laws are likely to be amended during the early years of implementation. A single act, legislating the NQF, the qualifications agency, or standards and vocational qualifications, often proves a blunt instrument. Education or labour laws need to be adapted as well, to integrate the principles of the qualification system reforms.

Effective stakeholder dialogue is about making sure that all are committed to making better qualifications, and are engaged in the necessary processes. This doesn’t mean getting as many organisations as possible involved, but making sure that all those who need to be involved can participate, understand what is expected from them, and know how to contribute. Stakeholder involvement can strengthen ownership and relevance of qualifications and their acceptance in both the labour market and the education system. Stakeholders can be involved at different levels, in setting policies or in implementation. It is important to note that the private sector is the main motor for employment growth in partner countries, even if the public sector remains an important part of national economies. Generally, the participation of the private sector in qualification systems is weak. The problem is often recognised by public actors, who show readiness to legislate, organise, and even subsidise private sector involvement. The main challenge is to engage representatives from the private sector effectively in a structural capacity to work on improving qualifications. Another essential group of stakeholders is education and training providers. They can become the main obstacle to system-deep reforms if they have not been engaged in the process.

The responsibilities and possible institutional arrangements that can support effective implementation need to be clarified, reviewing both existing institutional capacities and the need for additional capacities. In some cases, this will include creating new, specialised institutions for coordination and quality assurance, or for developing, assessing, or awarding qualifications. Institutions are needed to ensure a professional process for the development and use of qualifications; to organise the involvement of stakeholders; and to coordinate between different actors at different levels. In so doing they can empower the developers and users of qualifications to fulfil their functions effectively, and to externally quality assure the work performed by different actors so that qualifications are trusted.

The main function of quality assurance is to provide more confidence in qualifications and the competences of people who hold qualifications. Quality assurance focuses in particular on two aspects; ensuring that all qualifications that are part of the NQF register are relevant and have value, and that all the people who are certificated meet the conditions of the qualification. Quality assurance of the qualification system in its totality also plays an important role in regularly reviewing the functionality of the arrangements, as priorities for implementing the NQF are frequently changing. The issue of quality is an integrated part of the system of governance, rather than a separate issue.

This is by no means a new insight, as the ‘regulatory’ approach always had within it the issue of regulating the qualifications and the actors involved in qualifications frameworks. Moreover, since the lack of trust in existing qualifications and arrangements is one of the main drivers for greater transparency, a stronger focus on learning outcomes, and the comparability of qualifications, quality has never been de-coupled from legal and institutional arrangements.

7. Conclusions and recommendations
At the heart of our overall rationale for getting organised is the belief that comprehensive, coherent systems produce better qualifications. This coherence can be achieved through the development of the four foundation components identified above, starting with legislation.

Recommendations

• Promote a common understanding of qualifications.
• Don’t stop at developing an NQF – they are a necessary but not sufficient condition for systemic reform.
• Different systems need to be fit for purpose, that’s why they are different. To learn from others, look at the commonalities rather than the differences.
• Review existing qualifications before you develop new ones.
• Consider whether all qualifications are fit for lifelong learning.
• Make all qualifications available publically through an online database.
• Stakeholders from the world of work must have a role, as a prerequisite for systemic change.
• Recognise the inter-dependencies between actors in the system. No single actor can achieve change alone.
• Identify appropriate progress indicators and monitor them.

1. Key parts of legislation for better qualifications
2. The legislative process
3. Striking a balance between tight and loose legislation
4. Stakeholder involvement in the development of legislation
5. Ensuring that laws are implemented
6. Conclusions and recommendations

1. Key parts of legislation for better qualifications
To put it simply, countries use legislation to regulate things they want to change. The primary aim of legislation is then to specify what is going to be changed, who is in charge, what resources are available, and how responsible bodies are held accountable for what they are doing through monitoring and reporting.

We researched legislation in eleven countries, and found a wide range of qualification-related laws and by-laws. Our scrutiny of these countries’ legislation shows that many start with developing and regulating a national qualifications framework; others start with establishing a qualifications institute; and then there are countries that create a new qualification system outside the education system. For example, Turkey and Estonia created new qualification systems based on occupational standards. For Turkey, the legal lever for this reform was the law on the Vocational Qualifications Authority; while for Estonia, it was the Occupational Qualifications Act.

We also looked at the direct impact of the legislation, and asked some basic questions: Has the law been implemented? Has it achieved its purpose? Has there been a knock-on effect on other laws? And, for laws already in place for a couple of years, has the law improved the quality of qualifications and changed the lives of individuals? If the law has not been implemented, why? What is affecting or blocking its implementation?

In very general terms we can conclude that laws addressing institutions and new types of qualifications have more direct impact than a law on the NQF. But our main lesson learned is that reform processes aimed at better qualifications require eight key parts of legislation that cannot be isolated from each other. Key parts 1 to 3 regulate the foundations, while key parts 4 to 8 regulate different aspects of qualifications.

Key part 1: Regulating purposes and principles

The purpose of a law answers the question, what do we want to achieve with this law? The principles describe the contextual base of a law and answer the question, why do we need this law? Purpose and principles can be limited to the direct topic of the law.

For instance, the purpose of a law on a national qualifications framework will be to regulate the structure (the levels and descriptors, and types of qualifications included); institutional arrangements; and quality assurance. The principles of such a law might be to promote lifelong learning and to match qualifications with labour market and societal needs.

A law can cover a wider range of purposes and principles, positioning a qualifications framework or qualifications authority within a reform agenda. Ideally, the purposes and principles of a new law are based on a national strategy that has been defined and agreed in dialogue with a wide group of stakeholders.

Key part 2: Regulating institutional arrangements

To be implementable, each law should have a section on institutional arrangements that regulates the roles and responsibilities of the competent bodies, and identifies the resources to execute the provisions in the law. For example, the Kosovo Law on National Qualifications (2008) regulates the status of the National Qualifications Authority (NQA) as an independent public entity, the membership of its Governing Board, the main procedures of its meetings and decision-making, and its management and reporting provisions.

The law defines that the NQA is responsible for the development of policies and strategies for the establishment and implementation of the national qualification system. The law also defines a range of functions of the NQA in regulating the NQF (including design and approval), and in regulating the awarding of qualifications. The law stipulates the responsibilities of the NQA as follows:

• Establishment and maintenance of a comprehensive qualifications framework
• Regulation of the awarding of qualifications in the framework, with the exception of qualifications which are regulated under the provisions of the Law on Higher Education and qualifications explicitly regulated under the provisions of other legislation.

Key part 3: Regulating stakeholder involvement

Laws can regulate the roles and tasks of stakeholders in implementing aspects of a qualification system, as part of its institutional arrangements, as the following examples demonstrate.

The Occupational Qualifications Act of Estonia (2008) delegates many decision-making rights and responsibilities in the occupational qualification system to professional and sectoral bodies. The Vocational Education and Training Act of Lithuania (first issued in 1997) stipulates the establishment of sectoral professional committees as tripartite bodies responsible for the approval of sectoral occupational standards. The Implementing Regulation on the Establishment, Duty, and Working Principles of Sector Committees of Turkey defines the procedures for the establishment of sector committees, their governance and work procedures, and their functions. It foresees sector committees as collegial multipartite entities providing counselling, and executing review and quality assessment of occupational standards. Sector committees will provide the expertise and feedback of sectoral stakeholders in a more centrally governed national system of qualifications.

But legislation can also be an obstacle to stakeholder involvement. Trade union representatives in Tunisia have indicated that a crucial reason for slow progress in their country’s national qualification system reform is that it is still based on legislation from the pre-revolutionary period. For example, according to the current legislation, continuing vocational training (CVT) is an object of government regulation, while employers and trade unions have no rights to define the contents and organisation of provision of CVT. This issue is widely discussed amongst policymakers and social partners, and different solutions are proposed, such as the introduction of an individual right to CVT and the recognition of informal and non-formal learning. These ideas cannot be implemented until the current legal basis is changed.

Key part 4: Regulating development of qualifications

Regulating the development of qualifications is aimed at improving their quality, making qualifications comparable, and introducing national standards and learning outcomes to ensure relevance for the labour market and society.

Regulations concerning the development of qualifications are generally part of laws with a broader scope; for instance, laws that regulate a qualifications authority and an NQF, or a wider VET law. These laws regulate the principles of the design and development of qualifications, introducing national occupational standards. Examples include the respective laws of Kosovo,Turkey, and Lithuania.

More detailed provisions regarding methodologies and requirements for the approval and updating of occupational standards are regulated in by-laws (secondary legislation).

Legislation about the development of qualifications normally refers to vocational qualifications. Higher education institutions are autonomous and develop their own programmes, which are subject to quality assurance via HE accreditation processes.

Key part 5: Regulating the national qualifications framework

A national qualifications framework brings order to the landscape of qualifications, cutting across the entire education and training system. Therefore, one overarching NQF law should regulate the main features of the NQF. Many countries have a separate NQF law or decree, while in some the NQF is part of a broader law that also regulates other parts of the qualification system (as in Kosovo, France, Hong Kong, and Estonia). Increasingly, NQFs are integrated into legislation of educational sub-systems, such as in Albania’s new laws on HE and VET (draft), which both refer to the Albanian Qualifications Framework.

The features of a national qualifications framework that should be regulated include:

Scope. Which education sub-sectors and types of qualifications are included in the NQF? Are qualifications that are not the outcomes of formal education included?

Structure. That is, the levels and level descriptors in the NQF.

Management both of the NQF itself and the implementing institutions.

Database. A register or database of qualifications, and its link with the NQF. Does the register/database only contain qualifications that are included in the NQF or other qualifications – for example, legacy qualifications, that are not any longer awarded, but held by many people in the labour market?

Relationship with other instruments. Is the NQF the national instrument for structuring and classifying qualifications in a country, or are there others? How are different classification systems aligned? For example, former Soviet countries like Belarus and Kazakhstan are developing new NQFs while they have an existing qualification system (the former Soviet tariff-based qualification system; see Chapter 1) which guarantees access to further learning, to jobs, and to career development, salaries, and pensions.

Access. To qualifications, and to the horizontal or vertical progress between qualifications and qualification levels, and to the transfer of credits.

Learning outcomes. The basis for qualifications.

Quality assurance. Both of the qualifications in the NQF and the framework itself. What are the procedures for inclusion of qualifications (see Chapter 4)?

VNFIL. Validation of non-formal and informal learning (see key part 7, below).

EQF. In other words, linking to the wider European dimension allowing transparency, mobility and comparability.

Key part 6: Regulating quality assurance of qualifications

This means regulating the processes to maintain the quality of qualification standards, assessment, and certification. It also includes regulating the bodies that are responsible for quality assurance of qualifications, and the coordination between these bodies. But quality assurance of qualifications also refers to procedures and criteria for the inclusion of qualifications in an NQF and database or register. All laws that regulate parts of qualification reform should have a section on quality assurance, as illustrated by the following examples.

Law on the Albanian Qualifications Framework, Albania: “A qualification is awarded when a competent body decides, by means of a quality assurance assessment process, that the individual has reached the specified standards.”

National Qualifications Law, Kosovo: “Carry out external quality assurance of assessments leading to the award of qualifications in the NQF; implement internal quality assurance of assessments leading to approved qualifications, to ensure consistency in the application of standards.”

NQF Law, former Yugoslav Republic of Macedonia: “Quality Assurance in the application of the Macedonian Qualifications Framework applies to the quality of qualifications in the Framework, the procedures that lead to acquiring qualifications, and the degrees, diplomas, credentials, and certificates that are awarded to the participants who have acquired the qualification.”
The decree enacting the Turkish qualifications framework (TQF) has an extensive section on quality assurance. One of the fundamentals of the TQF is to ensure effective cooperation among the bodies that are responsible for the quality assurance of qualifications. The article on quality assurance of qualifications states: “All quality assured qualifications that have been acquired through education and training programmes as well as other ways of learning shall be included in the Turkish Qualifications Framework. Criteria for ensuring the quality assurance of the qualifications shall be determined by the Authority.”

This is followed by articles about responsibilities for quality assurance for different types of qualifications: those under the responsibility of the Ministry of Education; in higher education; under the responsibility of the Vocational Qualifications Authority, in VET; and other qualifications. The section on quality assurance ends with an article about the qualifications database: “With the qualifications being included in the framework, a Qualifications Database where all the qualifications agreed to be included in the framework are officially recorded and information regarding the qualifications is stored shall be created, and it shall be regularly updated by the Secretariat.”

Key part 7: Regulating validation of non-formal and informal learning (VNFIL)

Validation of non-formal and informal learning allows individuals to demonstrate what they have learned outside formal education and training, so that they can use it in their careers and for further learning.

The EQF recommendation of 2008 speaks about validation of non-formal and informal learning in general terms, and recommends that member states promote non-formal and informal learning. The EU recommendation on VNFIL of 20126, however, sets specific goals for EU members, stating that:

“[T]hey have in place, no later than 2018… arrangements for the validation of non-formal and informal learning which enable individuals to:

(a) have knowledge, skills and competences which have been acquired through non-formal and informal learning validated(…);
(b) obtain a full qualification, or, where applicable, part qualification, on the basis of validated non-formal and informal learning experiences(…)”

This requires a link between the VNFIL arrangements and the qualifications in a country’s NQF. In 2014 Cedefop explored this link in its European inventory of VNFIL, which includes 36 reports from 33 countries, including non-EU countries Iceland, Liechtenstein, Norway, Switzerland, and Turkey. Separate reports cover the French-speaking and Dutch-speaking communities in Belgium, and the UK nations of England, Scotland, and Wales. Cedefop divides countries into three levels:

High: Learning acquired through non-formal and informal means can be used to acquire a qualification in the NQF and/or can be used to access formal education covered in the NQF (19 countries in 2014).
Medium: A link between non-formal and informal learning and the NQF is under discussion (17 countries in 2014).
Low: There are no discussions on the establishment of this link (0 countries in 2014).

In France, validation is completely integrated in the NQF; a qualification can only be registered in the national qualifications framework (repertoire national des certifications professionnelles) – which is the basis of the NQF – if it is open to validation. In all four nations of the UK, the link is also tight. In many countries the link is under discussion. Some of these countries do not yet have an operational NQF in place, nor arrangements for VNFIL. If they wish to follow the EU Recommendation on VNFIL, ETF partner countries8 should include provisions about VNFIL in their NQF legislation and include provisions about the NQF in their separate VNFIL legislation.

Key part 8: Regulating recognition of qualifications

The terms ‘validation’ and ‘recognition’ are often used interchangeably, yet they have different meanings. Validation refers to the process of confirmation of an individual’s knowledge, skills, and competences; recognition refers to the external recognition of that qualification – in other words, the piece of paper issued to that individual. Two types of recognition are relevant for qualification system legislation.

1. Recognition of foreign qualifications for regulated professions. Based on the EU Directive 2005/36/EC on the recognition of professional qualifications for regulated professions, EU member states need legislation to ensure smooth and unequivocal recognition of foreign qualifications in regulated professions. The scope of such laws is limited. It stipulates the norms and procedures for recognition of professional qualifications acquired in foreign countries.

2. Recognition of higher education qualifications (for non-regulated professions), under the Lisbon recognition convention. The Lisbon convention of 1997, developed by the Council of Europe and UNESCO, has been ratified by most European countries. The convention requires that holders of qualifications issued in one country shall have adequate access to an assessment of these qualifications in another country. The convention defines detailed procedures and arrangements for the assessment and recognition of qualifications. No such convention yet exists for VET qualifications.

2. The legislative process
Now that we have seen which key parts require legislation in a systemic reform process towards better qualifications, let’s look at the legislative process, which is framed by a series of core questions: Where to start? How to align old and new legislation? How to link framework laws to more specific laws? And how to ensure coherence between qualification systems, education and training systems, and the labour market?

Reforming qualifications involves many issues: developing qualifications based on occupational standards; involving the world of work in qualifications development; introducing quality assurance of qualifications alongside quality assurance of providers and programmes; establishing a national qualifications framework to create order and transparency in types and levels of qualifications; and creating a database to make information about qualifications accessible to the public.

Such reform processes can take up to ten years and are difficult to plan in a linear fashion. Does it matter where you start the reform process and when you start with legislation? You have to start somewhere! As a basic rule, you can legislate when key stakeholders have a common agreement on the direction of the needed changes. Therefore, the best advice we can give is to start with a strategy for qualifications reform. This strategy should analyse the main problems you want to solve and what is required to solve them. Is there a lack of trust in qualifications by end users? Or a mismatch between the supply of qualifications and demand from the labour market? Or are lifelong learning opportunities blocked by a lack of access and permeability of the learning opportunities that lead to qualifications? If there is consensus about the main problems and solutions, you can prioritise and plan actions in different stages. Legislation can be built from there; the first legislation could focus on something that has an immediate impact. Legislation is often a prerequisite for making things happen, so don’t delay any necessary legislative process.

Laws are not forever

Laws are made for the future, but are not forever. Georgia, Moldova, and Ukraine are now reviewing and completing NQF laws that were developed to start qualifications reform a couple of years ago. While these NQF laws played an important role at the beginning of the reform of individual qualifications, they have not contributed to clarifying the relationships between different qualifications and bringing order into the framework. These countries are now reformulating their NQFs to make them more functional, and to clarify relationships between qualifications at different levels.

The overall approach that is taken and the chosen starting point are not the decisive elements in implementing qualifications reform. Instead, the decisive elements are, first, that there is an identified need for qualifications reform; second, that different pieces of legislation are consistent with each other; third, that what is to be regulated is realistic; and – perhaps most critically – that all of this can truly be implemented.

Aligning old and new legislation

Every country has an existing legal framework, so it is not possible to start from a blank slate when legislating qualifications reform. A major challenge is ensuring that old and new pieces of legislation are aligned. Old and new laws often co-exist for a period of time, regulating different components that function in parallel with each other. This becomes problematic when laws are contradictory, creating overlapping competences in some aspects and ‘empty spaces’ in others. Consistency of legislation is especially important for implementation. Fragmented legislation makes arrangements unclear for local actors who have to implement them.

There are two main options when aligning old and new legislation; a country can either adapt existing laws by making amendments and constructing new by-laws, or it can develop a completely new legislative framework. To decide whether to adapt existing legislation or draft new legislation, you should have a good overview of the existing legislation. This requires a mapping exercise of all relevant laws and by-laws. The mapping should include an analysis of which pieces of legislation support the reform and which are contradictory. A decision can be made about restructuring the legislation, based on this analysis.

In 2015 Albania started drafting a new VET base law, after mapping the existing VET legislation. The new VET Law, after adoption in 2016, will replace the old law which dates from 2002. This old law has been amended several times, but still has many restrictions. Too many by-laws accumulated over the years have resulted in fragmentation. Due to the many amendments and new regulations, it is almost impossible to keep an oversight. Different laws regulate VET and have created overlapping competences or contradictions in some aspects, and gaps in others. Despite the multitude of regulations, many things remain unclear for the7 local actors who have to work with and implement them.

The new VET Law will support current reforms in the VET system and will be aligned with the Law on the Albanian Qualifications Framework (AQF) that has recently been revised too. A package of secondary legislation for implementation will be added to both the new VET Law and the revised AQF Law. The new legislation is part of the action plan of the National Employment and Skills Strategy 2014-2020. (See also how stakeholders were involved in development of the new VET law in Albania, in section 4 of this chapter).

In Georgia, implementation of the NQF and related reform led to revision and redrafting of the existing laws. According to the representative of the Department of VET at the Ministry of Education and Science, one of the core motives of the recent changes of legislation is to address a misfit of legal acts with national strategies. For example, the existing Law on VET does not fit with the national strategy for development of VET in 2012-2020, which foresees implementation of open, inclusive, modern, and development-oriented vocational education.

Therefore, the Ministry decided to draft a new Law on Vocational Education and Training. This law is targeted at solving one of the major problems and shortages of the VET system, namely the absence of permeability between initial VET and higher education pathways. This makes VET a dead-end from a lifelong learning and careers perspective, because VET students cannot acquire secondary education and access higher education after graduation. Another important planned change is the integration of the current, rather separate, sub-frameworks of the NQF into one comprehensive qualifications framework. This responsibility is delegated to the National Centre of Education Quality Enhancement. Social partners are also actively involved in this process. (See also how stakeholders were involved in development of the new VET law in Georgia, in section 4 of this chapter).

Primary and secondary legislation

Legal arrangements differ from country to country, but usually start from the constitution that sets out the powers and functions of the parliament or assembly, the government, ministers, and so on. The constitution is the principal source of law. Constitutions will generally define the division between legislative and executive (and judicial) powers, distributing authorities among several branches. Legislative power is the authority to make laws. The legislative branch of government in parliamentary systems is the parliament or assembly. Laws that are adopted by parliament or assembly are primary laws, setting out general principles.

Executive power is the authority to enforce laws and to ensure they are carried out as intended. The executive branch of government includes the head of state (president) and/or the head of government (prime minister) and ministers. Most countries have primary laws on VET and higher education, adopted by parliament and signed by the president. Then the president, or the council of ministers, or the minister of education (or equivalent) will make detailed provisions through secondary legislation. This can be in the form of decrees, orders, by-laws, or regulations, with the exact title depending on the legal system of the country.

Summarised: Primary legislation sets out general principles and is adopted by parliament or assembly. Secondary legislation defines detailed provisions based on the general principle, and is the authority of the executive branch of government (head of state and ministers). For example: Turkey’s Primary Law on Vocational Qualifications Authority (2006, amended in 2011). Implementing regulations on the development of occupational standards and the establishment of sector councils, the amended law of 2011 set out the principles of the Turkish Qualification Framework (primary law). The TQF regulation with detailed provisions was adopted in 2015 (secondary legislation).

Kosovo’s Law on National Qualifications (2008). This law has a broad scope, covering the establishment of a national qualification system based on a national qualifications framework regulated by a national qualifications authority. The law is supplemented with a range of administrative instructions (secondary legislation) for detailed provisions. NQFs are mostly legislated by resolutions or decrees, but smaller countries like Albania and the Former Yugoslav Republic of Macedonia have adopted NQFs through primary laws. The amount of detail found in primary laws varies widely from country to country, as we shall see.

3. Striking a balance between tight and loose legislation
Striking a balance between tight and loose legislation is not easy. There are examples of very tight or rigid legislation, and examples of loose or even no legislation. Most legal systems are mixed systems, with some elements of both tight and loose arrangements. Which way the scales tip depends largely upon the balance of powers and the division of responsibilities between stakeholders in a country, and on its cultural heritage.

Typical examples of loose legislation are found in the English-speaking world, where governments have been less inclined to legislate (prescribe) what qualifications should look like. Initiatives to develop qualifications come from private actors based on the principle that ‘everything is allowed unless it is forbidden’. The market regulates the number and quality of qualifications. High value qualifications are the result of actors in the market acting in freedom and looking for the optimal way of defining qualifications. Qualifications compete with each other, and consumers will choose those that offer the best value for money.

Not surprisingly, the construct of qualifications frameworks originated in Anglo-Saxon countries to regulate this free market of qualifications. The UK introduced qualifications frameworks to help employers compare the many hundreds of qualifications available. Currently the UK has five qualifications frameworks that together accommodate the majority of qualifications in use in the various education, training, and lifelong learning sectors.

A typical example of loose legislation is the Scottish Credit and Qualifications Framework (SCQF), which uses common principles set out in a handbook. Although its constituent parts include regulatory frameworks, the SCQF is a voluntary framework. It uses two measures, SCQF Level and Credit Points, to help with understanding and comparing qualifications and learning programmes. Another example of loose legislation is the Framework for Higher Education Qualifications (FHEQ) in England, Wales, and Northern Ireland, with the Quality Assurance Agency for Higher Education (QAA) as the external quality assurance body. In the FHEQ, universities are responsible for developing their own qualifications and may use their own approaches as long as they can justify them. Loose legislation fits in the Anglo-Saxon Common Law tradition. Legislation is built incrementally around individual cases that are generalized to a larger area, creating precedence.

At the other end of the scale we see examples of tight legislation that come from the other main legal tradition, Civil Code, a system of state regulation built on general principles that are logically extended into a systematic collection of laws. Tight arrangements are aimed at developing high quality qualifications and making these compulsory for users. High value qualifications are the outcome of consensus and cooperation between actors, rather than competition. Qualifications frameworks in countries with tight arrangements are not intended for regulating the free market of qualifications, but for establishing principles for high value qualifications, such as ensuring conditions for equity and access to quality for all.

France established an NQF as part of the Social Modernization Law of 2002. The purpose of the law is to create social mobility for citizens by enabling reforms in the fields of health protection, social security, employment, and vocational education and training. The law regulates the French national system of qualifications and its main instrument, the National Repertory of Qualifications (France’s national qualifications framework). The law also introduces instruments for the validation of non-formal and experiential learning. According to the law, each person active in the labour market has a right to have his or her skills and competences validated, whether acquired through experience in professional, non-professional, or voluntary activities. Validation can be total or partial, and successful candidates receive the corresponding certificate or diploma that is included in the list of qualifications established by the commissions of sectors of economy and registered in the National Repertory. The law regulates that employees have a right to paid leave designated for validation of experiential learning outcomes.

Another example of tight legislation is the Ausbildungsordnungen in Germany that have the force of law and are agreed between social partners and the Federal Government. The main idea behind this is ‘social engineering’. Ensuring that learners have access to well- defined and agreed broad-based qualifications means they will be better equipped for employability. The youth employment figures in Germany suggest that this works, although there is of course also an important element of pathway choice.

While loose arrangements are typically found in self- regulated systems, tight arrangements are found in state- regulated systems that are driven by social dialogue. Tight arrangements supported by legislation can provide a greater degree of centralised quality control, and hence provide assurance that all qualification holders meet common requirements of a unique national standard. In these systems, it is easier to link the qualification to specific legal rights related to salaries, pensions, and social dialogue.

In most ETF partner countries, central governments tightly regulate the roles of the many actors in their qualification systems. A 2009 Cedefop study refers to this as the ‘prescriptive model’9. Many ETF partner countries are in transition from centrally-planned to market-based economies. They have state-regulated qualification systems, but social dialogue is weak. Countries lack experience with markets as well as consensus structures, because in the past the state decided what was good for everybody. The legal systems of countries in transition are often weak, with laws that exist on paper but are not implemented and often not known or respected by the public. A legal compass is missing.

Countries in Central and Eastern Europe seek a replacement for the former socialist law system, and going back to pre-World War II laws is not a solution because they are no longer fit for a modern economy. New laws are being developed. Governments have a central role and use legislation as a catalyst to initiate and implement qualification system reform, but the reputation of existing arrangements casts a shadow on the reputation of new laws. There is less trust in legislation. Tight legislation is often seen as a guarantee against corruption, but if many diploma mills can exist in a country then who will trust the letter of the new law? Laws cannot create stability where there is total chaos, but they can strengthen a society and may be necessary to ensure that a state monopoly is abolished, giving the right to third parties by decentralising or devolving responsibilities. However, to be effective laws need to be accompanied by institutional reforms. (See Chapter 4).

Legislation should increase trust in qualifications

In countries where trust in qualifications is low, legislation can increase trust by defining explicit arrangements for the development of qualifications and assessment and certification, with strong regulatory bodies supervising these arrangements. Very detailed legislation creates a rigid system that is difficult to change and can lead to bureaucracy, while regulating too little can lead to legislation that is vague and creates ambiguities, and sometimes, is used as an excuse to not progress on implementation.

Legislation is also used to protect the workforce. For instance, Turkey has a new, highly regulated approach to increasing trust in qualifications and protecting the workforce. A new law (6645) was adopted in April 2015, making part of the NVQ system obligatory. Certification will be compulsory for, initially, 40 occupations that involve health and safety risks. Employers will have to make sure their employees are certified before May 2016. This concerns large numbers of workers, including 1.4 million employees in the construction sector alone.

Regulating professions

By regulating professions, countries restrict rights to practice a profession. Individuals are required by law to be qualified by a professional body before they are permitted to practice in a regulated profession. Some countries regulate only professions that have a high risk profile. Other countries regulate many or even all professions to ensure the quality of the practitioners of the profession and regulate access to a profession. Over-regulating professions creates a rigid labour market and barriers to mobility, and should be avoided.

Other examples include former Soviet countries with a relatively high level of wage employment, such as Belarus, Russia, Ukraine, and Kazakhstan, which have a tendency to make tight arrangements building on the regulatory system that existed in the Soviet era (see Chapter 1).

In many legal frameworks there are tightly regulated parts, in particular for vocational qualifications and for general secondary qualifications, such as the qualifications regulated by SQA in Scotland, or the qualifications in Turkey that are part of the National Vocational Qualifications System, or the qualifications in Ireland regulated by QQI.

In Turkey, legislation for the Vocational Qualifications Authority defines how occupational standards and national vocational qualifications are developed, and how they can be awarded through authorized certification bodies or VocTest Centres that are also accredited against the ISO 17024 standard.

4. Stakeholder involvement in the development of legislation
As noted above, it is possible to legislate when key stakeholders have a common agreement on the direction of needed changes. The explicit involvement of stakeholders should be a strategic decision, not an afterthought. The focus here is on stakeholder involvement in the development of the legislation, the process that extends from defining the content of primary and secondary legislation to the adoption of the law and approval of secondary legislation. Stakeholder involvement in the implementation of the legislation, meaning enforcing the law and executing what is stated in the law, will be elaborated in Chapter 3.

Three models of stakeholder involvement in developing legislation

In most cases, the ministry of education initiates a new law or the adaptation of existing legislation. Drafting a law or amendments to a law is a technical process that requires professional and legal expertise. But defining the content of a law cannot be left to a small group of experts. The involvement of all stakeholders who will be affected by the law, and who will have a role in its implementation, is essential to reach a shared understanding of principles and commitment. We can distinguish between three models in the way stakeholders are involved, with declining effectiveness (although in practice the dividing lines are not always so clear-cut).

I) Legislation based on a shared strategy

In the ideal scenario a new law is based on a strategy, with an action plan, which is developed and adopted in an effective process of stakeholder dialogue. If this is the case then the principles and purposes of the law are already defined in the strategy and action plan. The content of the law will develop these principles, purposes, and institutional arrangements for the topics of the law, such as regulating a qualifications authority and/or a qualifications framework. As stakeholders have already been involved in defining the strategy, specialists can have a bigger role in the actual drafting of legislation, while stakeholders are consulted before a law goes to parliament.

II) Creating a shared vision during the legislation process
When there is no shared strategy and no shared vision for the future, such a vision must be articulated in the process of legislation development. This requires active stakeholder involvement in the legislation process itself. The private sector may even take the initiative to propose draft laws. This happened in Ukraine, where the focus of the Ministry of Education is on improving education and training through better provision, while the focus of employers is on a better- qualified workforce. The employers have proposed several laws that failed, but structural changes may materialise soon. A new National Framework Law on Education, is currently completed replacing the 20-year-old education law. The purpose of this new law is to regulate the education system for the future and create a framework for a lifelong learning system. Ukraine had already introduced a national qualifications framework in 2011, but it lacks a wider legislative basis. There has been a two year process to decide what should be written in the new law and how to use the framework of qualifications to support lifelong learning, new pathways and quality assurance beyond the formal education system.

It started with an Expert Round Table10 organised jointly by the Committee on Education and Science of the Ukrainian parliament (Verkhovna Rada), the ETF, and the Council of Europe. Different positions were taken by the Committee on Education and Science focusing more on the opportunities of citizens and the Ministry of Education which was more concerned with the needs of the education system. The representatives of employers’ federations were fully engaged in the process, emphasising more attention to adult learning, relevant qualifications and independent quality assurance mechanisms. The law has been discussed in several hearings in the parliament, but adoption is still uncertain.

III) Drafting legislation and then seeking the approval of stakeholders
When stakeholders’ involvement is weak their influence on legislation will also be weak. In such a setting it is mostly public authorities who initiate legislation and carry out most of the drafting work. Representatives of industry, providers, and NGOs are consulted about draft versions of the law and their feedback is accommodated in the final draft that goes to parliament for adoption, or to an executive authority for approval. Consultation usually takes place at a later stage of the drafting process, in a number of public hearings that are organised after the draft law has been discussed in the ministry. In fact this model is more about validation of stakeholder opinion rather than involvement. Stakeholder involvement has to come from two sides. The government should take stakeholders’ opinions seriously and allow their involvement. For representatives of industry in particular it would be better to consult them at an earlier phase of legislation development, as every piece of legislation that is related to increasing trust in qualifications involves both the national education and training system and the labour market.

Experience shows that individual employers are deeply involved in the day-to-day demands of their businesses, and do not have the time or the expertise to participate in the development of legislation for better qualifications. But, in any case they can always be consulted through their organisation/sector representatives. The active involvement of industry requires that its representatives are organised in employers’ federations, sectoral federations, a chamber of commerce, and trade unions. These bodies can then put issues related to good quality qualifications on their agenda, and set up dedicated working groups that have the time and expertise to participate in the development and implementation of qualifications reform (including legislation). An emerging route in many countries is the creation of sector skills councils, who can take on the role of expressing the sectoral point of view in qualifications reform, including the needs for new and/or revised legislation, amongst other issues.

In Georgia, the drafting of the new law on VET started with the establishment of a working group of about 20 experts coordinated by the Ministry of Education and Science. This group elaborated the issues and provisions of the law. A smaller group of legal experts prepared the first draft of the new law, based on the outcomes of discussions of the working group. The draft law will be discussed in parliament in 2016. Representatives of Georgian employers’ organisations recognise that they play a rather passive role in the development of VET legislation. Social partners are consulted only at a later stage in the drafting process, when there is a public discussion.

International donors’ role
Partner countries sometimes involve international donors in the development of legislation. External experts can speed up processes as they bring knowledge, experience, and funding. However, they will always work in national contexts that will ultimately define the progress made. A shared national strategy should be the basis, and external experts should work together with government and other stakeholders. When governmental and stakeholder commitment to their work is lacking, or decreases because of internal disagreements or changing political realities, then external experts risk working in isolation and their seeing their work end up on a shelf in a government office.

Some modern draft laws on vocational qualification systems developed in the scope of EU projects were rejected by government, because the gap between the draft law and the practices of a traditional system that is still largely input-based was just too big. A lesson learned here is that you cannot impose laws. Another consideration is that foreign concepts do not translate easily into some official national languages. However, there is no problem in adapting international terminology into local terms for better understanding and implementing reforms.

The Albanian Council of Ministers adopted an Employment and Skills Strategy 2014–20, with a concrete action plan that foresees the overhaul of VET legislation. A new VET law aligned with the principles and purposes of the Albanian Qualifications Framework has been prepared with ETF and donors’ support. The Ministry of Social Welfare and Youth (responsible for the VET system in Albania) established a wider group of experts representing Ministries, the agency for VET and Qualifications (NAVETQ), and the National Employment Service. A series of workshops was organised to discuss the principles and various sections of the new law with this group of experts. After consultation with stakeholders the draft VET law is now adopted by parliament. Currently, new secondary legislation (by-laws and regulations) is being drafted.

5. Ensuring that laws are implemented
A law is just words on a piece of paper and unless it is properly and timeously implemented, it will not make any difference. By ‘implementation’, we mean that the law is enforced, and that what is promised in a law is executed. A number of conditions are required to make this happen.

Political change
Political change is a fact of life and democracy; new ministers come, policies change, old and new agencies work in parallel to each other. The challenge is to make certain that change moves fast, not only when the wind is in the right direction but also when the wind changes direction. The answer is to ensure that qualifications reform is embedded in wider reforms, that reform is supported by donors and state agendas, and that stakeholders are involved.

It is important to communicate reform plans to new decision-makers and create links to their priorities. This can create conditions for the reform to continue after political change. It is equally important to focus on results and mobilise people behind reforms, including end users such as the learners and their families, as well as employers’ and workers’ representatives.

Unambiguous language
A law should be formulated in such a way that the law can be executed, therefore the language used in the law should be unambiguous. Lack of clarity in a law can result from the careless use of language, ambiguity about what should be in the law, or a lack of agreement about what should be in the law. Vague descriptions in a law seldom lead to concrete actions. Legal experts can help with writing a law in unambiguous language only if it is crystal clear to them what should be described. A fundamental question to ask in consultations about draft laws is whether the law is absolutely clear.

Consistency with other laws
If a new law is contradictory to other laws then the new law cannot be implemented. If this happens, it is bad for trust in the new law and also for trust in legislation in general. Therefore, when drafting a new law, or revising an existing law, identify other relevant laws and look for consistency with them. For example, Albania is currently revising the NQF law that was first adopted in 2010 but was never properly implemented. Other relevant laws that will be taken into account to create consistency are the new law on higher education (adopted in 2015); the new draft VET law (expected to be adopted in 2016); a new law on Crafts (to be adopted in 2016); and the new Labour Code (2015). When introducing reforms, a new law will replace old laws and regulations that are no longer valid. In which case, an article is included in the new law stating that this law has priority over other related laws.

Similarly, consistency can be a problem in countries which quickly developed new laws when the former Soviet system ended. Direct cross-referencing is not possible in some countries legal systems, where there is just reference to ‘other legislation’. This might include education laws as well as laws relating to finance, property, labour, and the legal status of HE and VET institutions.

An institutional home
To ensure that provisions in laws are executed, a law should have a section on institutional arrangements and the responsible authorities within these arrangements. This should clarify the division of responsibilities and competences between different actors by identifying which bodies are responsible for which functions, and what their core competences and duties are. This creates an institutional home, and establishes the conditions for the different actors in the qualification system to work together effectively. Chapter 4 gives the details.

Operational arrangements
Primary laws, defining general principles, should be supplemented with secondary legislation defining the operational arrangements and resources for implementation. The amount of detail in secondary legislation will vary per country. Over-regulation might lead to bureaucracies that work against effective implementation. Provisions for funding the tasks that have to be executed, and the bodies that have to execute them, should also be regulated in secondary legislation.

Stakeholder involvement and dialogue
The roles and tasks of stakeholders in implementing parts of a qualification system can be regulated by law as part of institutional arrangements. But for proper stakeholder involvement a continuing dialogue is required to share principles and commitment, so that all actors can deliver on their tasks. This is discussed further in Chapter 3.

6. Conclusions and recommendations
Laws create the setting in which better qualifications can be designed and delivered. The passing of legislation is a necessary but not sufficient condition, and it must be implemented to achieve the desired results.

Recommendations

• Don’t delay any necessary legislative process.
• Ensure legislation covers the key functions identified in this chapter.
• Ensure new legislation is based on an agreed strategy for reform.
• Map existing legislation to identify what needs to be done.
• Make sure new and existing education and labour market legislation is aligned.
• Enact regulation that empowers actors, rather than seeks to control them.
• Use primary legislation to establish principles, and secondary legislation for operational functions.
• Consult/involve stakeholders when drafting legislation.
• Regulate stakeholders’ involvement in policy, design, and implementation, and remove legislative obstacles to that involvement.
• Don’t design laws that cannot be implemented.
• Allocate resources and institutional capacities to implement laws.

1. The importance of stakeholder involvement
2. Stakeholders in a qualification system
3. Influences on stakeholder engagement
4. Stakeholder mapping
5. Distinguishing between dialogue platforms and implementing bodies
6. Engaging with stakeholders
7. Ensuring stakeholder coordination
8. Conclusions and recommendations

1. The importance of stakeholder involvement

As we have seen, there is a lack of confidence in qualifications in many partner countries. Qualifications are often seen as not relevant, or not understandable, and do not adequately capture people’s competences. Some think qualifications do not matter; ‘people need skills, not diplomas’ is a frequently heard view.

Mobilising the relevant stakeholders to support the reform of qualifications and the VET system should lead to better qualifications, more engagement with vocational education and the VET system, and better outcomes for individuals. But it is not only a matter of more stakeholder input in the process. Stakeholders can gain as well, as they extend their influence on education and training systems, making sure these meet their needs and those of the groups they represent. While Ministries of Education represent core public interests and basic educational requirements, the involvement of different stakeholders can enrich the outcomes of education bringing them in closer contact with changing social, economic and technological demands. The mobilisation of stakeholders can thus support the development of a ‘zone of mutual trust’. This is described as a stakeholder agreement covering “the delivery, recognition and evaluation of vocational learning outcomes (knowledge, skills, and competences)”.

The term ‘stakeholders’ will be familiar to readers, but it is worth noting the difference between ‘stakeholders’ and ‘actors’. Stakeholders are people, groups, or entities that have a role and either a specific, or a general, interest in the objectives and implementation of qualification policies. Actors, on the other hand, are authorised agents for particular interests – autonomous entities who can exercise agency (the ability to effect change) in a given situation; in this case, the development and implementation of qualification policies. Not all stakeholders are actors, but all actors are stakeholders, so we will mostly use ‘stakeholders’ in this discussion since it necessarily includes actors.

The value to qualifications of engaging stakeholders

Different types of engagement can lead to different outcomes. It is more difficult to reach agreement on the outcomes when different stakeholder groups are involved, but a more inclusive approach to stakeholders, can produce better outcomes. A European study of bricklayer qualifications12 shows lower levels of qualification among English-trained bricklayers, in contrast to higher levels among their French, German, and Dutch counterparts. In France, Germany, and the Netherlands, bricklayer qualifications draw on a broad knowledge base, and their development is a product of dialogue among social partners. In England, qualifications are more on-the- job based and narrower in focus. Their development is not driven by social dialogue but rather by employers seeking quicker, less costly solutions, reducing the role of stakeholders whose involvement would otherwise lend credibility to the qualifications in question. As the study says, “Any occupational qualification depends for its validity on the involvement and agreement of all stakeholders; the less the agreement and involvement of all those concerned, the weaker its currency and status in the labour market is likely to be.”

2. Stakeholders in a qualification system
Various typologies of stakeholders exist, but in order to simplify matters we have identified five common groups. The first is those organisations representing public authorities, such as ministries and governmental agencies, but also regional authorities. The second group is organisations representing industry, including employer and employee representatives, and intermediate organisations such as chambers of commerce and craft, or organisations representing a specific economic sector. Third, the education and training providers. Fourth, individual learners, their families and communities. And fifth, international donors such as the British Council, GIZ, the EU, and the World Bank, amongst others. Non-governmental organisations (NGOs) or the now so-called civil society organisations (CSOs) may also be identified as stakeholders.

The ILO has set out a list of stakeholders in the development of national qualifications frameworks that may serve as a useful guide here. It extends, they note, from the “more obvious stakeholders” to include a call for open public participation in a consultation process that allows “any person or organisation” to be considered as having “self-identified as a stakeholder.

The list includes:

• Academic researchers working on education and labour force issues
• Awarding bodies and quality assurance agencies
• Career guidance professionals
• Community and voluntary organisations
• Education and training providers
• Educators of teachers and trainers
• Employer and worker organisations
• Government departments and agencies
• Learners and students
• Professional bodies
• Representatives of migrants
• Teacher and trainer staff associations.

Having identified these – and possibly others – as potential contributors to the overall development of the qualification system, gaining and maintaining their involvement can be aided by mapping, prioritising, and coordinating dialogue with them.

3. Influences on stakeholder engagement
The ideal scenario is to identify and involve all groups of stakeholders, and to put in place cooperation mechanisms to maximise the focus on good qualifications and wider lifelong learning policies. It is important to recognise the ‘entry point’ of the process; that is, the current state of play with stakeholders in a given policy context – including identifying those who are drivers of change – and start, or continue, building from there. The ETF guide developed in the Frame Skills for the Future project provides contextual considerations for the inclusion of stakeholders.

Sometimes the catalyst for change is a donor project, at other times change is motivated by an employer or employer group, or it can be led by local or central authorities. Whoever takes the lead, the task for other stakeholders is to work out how to support this, for instance by identifying gaps in measures taken by government to promote a positive and results-oriented process. In Tunisia, this led to the reactivation of the existing, multi-stakeholder Higher Council for Development of Human Resources, which is responsible for the implementation and development of the NQF.

As our experience shows, the drivers vary from context to context and from country to country. A list of influencing factors for stakeholder engagement is helpful to improve understanding of the context, and above all to support adequate measures or policies. So while there is a common ideal goal, there is no single best approach or method to maximise stakeholder engagement in qualification processes. When considering the factors that can aid or impede the development of effective stakeholder dialogue, it is worth remembering that there may be a simple lack of awareness on the part of stakeholders when it comes to qualifications and the potential opportunities to get involved. Once awareness is established, an initial factor is exploring how best to support and build on the initiative, capacity, and power of specific stakeholders. In the case of Ukraine, employers have been taking the lead, whereas in Georgia the government is setting up structures to facilitate stakeholder engagement.

The holistic approach of involving stakeholders in the full VET cycle, not only in qualifications development, would necessitate their participation at all stages – policy, design, and implementation15 (as in Figure 3).

An important transversal factor is the capacity of stakeholder organisations, and the technical competence of their representatives, in the different stages of cooperation mechanisms. Too much dominance by government might negatively affect the full engagement of other stakeholders. Stakeholder representatives’ power in the decision-making context is another issue to consider, for example in Algeria, which has seen a fragmentation of organisations representing employers. Allocating a specific function to stakeholders such as responsibility for developing occupational standards, as seen in Estonia and the Netherlands, can facilitate stakeholders in acquiring expertise and a permanent role in qualification systems. Finally, the impact of policy and capacity learning via international cooperation projects, usually sponsored by donors, can be beneficial to engaging stakeholders on a more permanent basis. Such projects can also be a starting point for labour market stakeholders to organise themselves to become established long-term actors, speaking with a stronger voice and influencing national policy, used in permanent and sustainable social dialogue platforms.

4. Stakeholder mapping
The value of stakeholder mapping for most of the groups identified here and particularly for employers, is separate to their involvement in the qualification system reform. It is a strategic management function that should be encouraged as generic good practice. It is also a dynamic process, not something to be done once and then filed away. Groups change over time, and their relative influence and interest in a particular issue changes too. Stakeholder mapping is therefore an ongoing need. In Chapter 4, we establish the importance of having a lead or coordinating body for qualification system reform. It is likely that this body will take responsibility for overall stakeholder mapping. That does not reduce the need, or indeed the benefit, for all parties to undertake their own mapping process. Moreover, our purpose in relation to qualifications is to encourage systemic change, and this cannot be achieved by one institution only and operating and deciding alone. You need to have allies, you need to build networks, whatever your position in the qualification system. You need to know who the other stakeholders are, and how you can engage them, in order to bring all together and make changes happen.

There is a vast literature on stakeholder mapping within the VET sector, and from other sectors and professional areas. Stakeholder mapping is a central plank of strategic communication, and since the purpose of mapping within qualification system reform is to promote dialogue and engagement – both arguably forms of strategic communication – the mapping process will be very similar across all sectors. Sooner or later you will need to communicate systematically with other stakeholders. When developing targeted stakeholder communications, the broad typologies identified below need to be examined in more detail. Types must be further analysed into specific groups and organisations, and even individuals where appropriate (e.g. a particular government minister, or academic, or philanthropist, and so on).

VET reform efforts around the world have yielded useful guidance on stakeholder mapping. A South African project on workplace training recommends that “a rigorous ‘stakeholder mapping’ exercise should be carried out at the outset”. However, it goes on to say that “it is also important to acknowledge that the balance, timing and extent of stakeholder involvement should be carefully considered, to avoid creating a stakeholder management process that is unnecessarily bureaucratic and cumbersome.”

An apprenticeship standards resource for employer groups in Scotland suggests three steps for getting started:

1. List any stakeholders or partners that you currently work with and identify both the current role that they play and their potential future role.
2. Add any stakeholders or partners who are not currently involved, but who have the potential to support your work.
3. Through consultation or direct contact, confirm the interests of these stakeholders and partners and signpost them in a mapping matrix.

Once the mapping stage is complete, any subsequent consultation process must recognise the need for trust among stakeholders. As we saw in the case of the English bricklayers, a top-down-only approach can inhibit trust, so there is much to be gained from combining top-down direction with consultative, bottom-up approaches. Transparency and participation confer legitimacy on a consultative approach, and therefore levels of trust will be higher. However, it is not the case that all stakeholders must be assigned the same priority. Apart from anything else, it would be impractical.

Moreover, it is not usually necessary to engage all stakeholder groups with the same level of intensity all the time. Developing a strategic plan about who you are engaging with, when, and why, offers a more sustainable path. The choice of priority will depend on the relative levels of interest and influence that are attributed to stakeholders, and to a process of assessing the issues that are most pertinent for the high priority groups (see Figure 3). Questions to ask might include which issues stakeholders most frequently raise, and whether these issues are relevant to the engagement objectives.

The appropriate engagement format can be decided in a process based on four phases:

1. Identifying: listing and mapping relevant groups, organisations, and people.
2. Analysing: understanding stakeholder perspectives and interests.
3. Mapping: visualising relationships, mapped to objectives and to other stakeholders.
4. Prioritising: identifying issues and ranking stakeholder relevance by likely impact.

Figure 3. Power/interest matrix

For instance, the materials available from online resource Stakeholdermap.com19 offer a useful starting point. The site provides models, templates, and techniques for gathering and analysing information about stakeholders in business communication and project management processes, most of which can be easily transferred to the area of qualification system reform. The Stakeholdermap.com e-book 4 Steps to Successful Stakeholder Management includes stakeholder identification methods such as mind-mapping, brainstorming, analysing previous projects, and reviewing organisation charts and directories.

5. Distinguishing between dialogue platforms and implementing bodies
Stakeholders from different organisations come together to share a platform for dialogue in various settings. Dialogue is about agreeing the direction of development; whereas implementation is about the technical work resulting from agreed actions. So, while stakeholders carry out dialogue with each other, institutions have operational and implementing responsibility. But in practice it is not always easy to distinguish between fora for dialogue and implementing bodies.

For example, in Estonia the labour market is divided into 16 sectors based on statistical classification of economic fields, and each sector is managed by a sector skills council. Institutions represented in these councils are nominated by government, and include employer organisations, trade unions, professional associations, education and training institutions, and responsible ministries. These councils discuss various proposals and opinions, and achieve a consensus among represented institutions on the development and implementation of the occupational qualification system for each sector.

Among other functions, Estonia’s sector skills councils are responsible for developing, updating, and approving occupational standards, and giving rights to awarding bodies to award professional qualifications. They approve procedures for awarding occupational qualifications, and set the fees for awarding and recertifying qualifications. Cooperation between the sector skills councils is coordinated by a Board of Chairmen of Sector Skills Councils. The Board decides on the allocation of initial occupational qualifications in the EstQF, explores the need to develop higher qualification levels, and approves the development of occupational qualification standards.

Formal platforms exist in a number of ETF partner countries: Turkey has sector councils as well as other qualification-related councils; Bosnia has a Council and Intersectoral Committee for QF; and Morocco has a National Commission for the NQF. In some partner countries, there are currently only informal platforms, for instance Ukraine, where dialogue is building on previous NQF working group activities, and Azerbaijan, where an ETF-organised dialogue addressed shortcomings in the composition of the official NQF working group.

Moldova has both VET and HE stakeholders involved in sectoral committees, along with public institutions and donors. Sectoral committees have a role in identifying which occupational standards and qualifications should be developed for the sector, which can be considered a form of sectoral dialogue. Their responsibility to develop occupational standards, however, is more of an implementing task. Developing a partnership around a central qualifications agency to coordinate stakeholders is one option to bolster engagement efforts led by ministries.

A coordinating agency can also be a neutral meeting place for stakeholders. This is the case of Portugal, for example, where the Sectoral Qualification Councils meet regularly for deciding on updates, revisions or deletions of qualifications registered and used as national standards.

6. Engaging with stakeholders
Dialogue provides the primary means for different forms of engagement among stakeholders, directed towards the development and implementation of qualifications, qualifications frameworks, or qualification systems. As a means to an end, dialogue requires effective methods and strategic purposes to achieve a particular goal. Different counterparts with diverse, or indeed shared, interests may pursue this goal from different angles.

Formal dialogue is conducted by institutionalised actors who fulfil the preconditions of autonomy and representativeness, and both formal and informal dialogue can occur within and between economic sectors, professional bodies, individual businesses, and other social partners. Informal dialogue includes, for example, some ETF workshops and meetings in partner countries. While the informal nature of these activities is valuable, since it allows people to express views and to use language that they might not use in a formal setting, they run the risk of remaining exploratory discussions with little or no concrete output. It is important to explain the agenda in an informal dialogue, and ensure an adequate level of productiveness, to maintain the willingness of participants to keep involved. Decision-making and influencing are good indicators of productive informal dialogue. These aspects of dialogue will be framed by the value different stakeholders give to skills and qualifications, which is linked to varying cultures in public affairs, the role of the state, and social dialogue.

All of which may influence thinking on practices such as consultation, mediation, lobbying, and negotiations on the design, definition, and accreditation of qualifications.

Therefore, possible topics for dialogue might include:

• Analysing problems in the existing qualification system
• Developing an implementation plan
• Formulating NQF levels and identifying qualifications that should be included
• Formulating the objectives of reform
• Identifying needs for new qualifications
• Reviewing institutional arrangements and capacities
• Specifying how qualifications should change
• Taking formal decisions on new concepts.

This involves identifying and engaging the appropriate stakeholders in the different topics. In practice, there are four formats of dialogue or stakeholder engagement among the identified groups; informative, consultative, cooperative, and decisional. In informative dialogue the public authorities only inform stakeholders about decisions taken in the field of qualifications (raising the question of whether this can be properly termed ‘dialogue’). In the case of consultative dialogue, stakeholders are consulted and their feedback may or may not influence decisions. The cooperative form goes further, and implies that the participating stakeholders are involved in the decision- making process as partners. Finally, the decisional form is where the stakeholders themselves make the decisions. The choice of format for stakeholder engagement is of course also linked with the topics for dialogue.

Countries in which there is meaningful dialogue between stakeholders produce qualifications that are trusted by stakeholders and beneficiaries. In the case of Ireland, bodies for governance and dialogue are in place under the Quality and Qualifications Ireland banner. Many stakeholders are involved, both in the process of policy development and in the implementation of qualifications. The goals of the various dialogue forms in any given country will determine which stakeholders are involved. Both the form and goals of a particular dialogue then determine the degree of stakeholders’ involvement. The different forms of dialogue can also be called ‘cooperation mechanisms’, whether or not these mechanisms have been confirmed in laws, decrees, or any other form of regulation directing roles, responsibilities, and resources (see Chapter 2).

In many cases these cooperation mechanisms are not only concerned with qualifications, but also with other issues related to VET, such as identification of relevant skills, quality of provision, attractiveness, work based learning, curricula development and so on. A useful overview of these cooperation mechanisms is available in the ETF report on Governance of Vocational Training in the Southern and Eastern Mediterranean.20 The dialogue can be focused on a particular area or sector, or regionally oriented. The sector skills councils in certain countries provide an example of this type of platform (see section 7 below). The scope and nature of these dialogue platforms varies considerably, covering many areas in addition to qualifications.

Three different stages of stakeholder engagement can be identified in the qualifications development process. Each stage has its proper goals, and dialogue forms or cooperation mechanisms. In addition, the role and type of stakeholder differs for each stage.

The policy stage is concerned with the functions, procedures, and regulatory context of qualifications, as well as funding and support mechanisms. For example, there may be a national committee composed of various types of stakeholders. In certain cases, one of the participating stakeholders might be an international donor. In Morocco, the national qualifications framework was developed in close cooperation with stakeholders, and in particular with the employers’ association. However, NGOs and unions, and other organisations representing employees, were not part of this dialogue.

The design stage is about establishing the learning outcomes and agreeing the assessment standards for qualifications. In the Netherlands, working groups at sectoral level – involving experts representing VET providers and social partners coordinated by legally recognised sector skills organisations – define learning outcomes and assessment criteria for specific qualifications. In this case, the degree of involvement is high.

The implementation stage is about the education and training programme leading to the qualifications, and the assessment of learning outcomes. Companies with an interest might have a role in the formal assessment of individuals; the well-known dual system in Germany is an example of this approach. The way stakeholders are involved in the dialogue and cooperation over qualifications can, in practice, differ within the stages and from country to country, or even within a country. It often depends on how the systems of dialogue are defined, and to what extent stakeholders have formal roles and responsibilities. In Egypt, Enterprise and Training Partnerships (ETPs) were created following a donor support programme.

These ETPs are organised according to economic sector, and at regional level. Their mission is to provide a bridge between the demands of industry and the VET system, but their involvement is based on unclear consultation mechanisms. At this stage, since there is no national qualifications framework in Egypt, these ETPs have no formal role.

7. Ensuring stakeholder coordination
Since responsibility for the quality of qualifications ultimately rests with government, it is reasonable for the stakeholder engagement process to start there. This, though, raises a series of questions: Is government serious about boosting stakeholder engagement? Is government open to bottom-up involvement? Should engagement be conducted directly between government and the multiplicity of qualification system stakeholders, or should there be some kind of filtering or aggregating mechanism? Within the public sector, the qualification system is often characterised by fragmentation, and stakeholders frequently complain of weak links among the various ministries responsible for youth, education (HE, VET and general education sometimes are under the responsibility of different ministries or ministerial departments), employment, and other relevant areas. Bodies that, on paper, offer the promise of playing a coordinating role among diverse stakeholders, such as inter-ministerial committees concerned with workforce skills, are often not playing that role very effectively, or not even meeting regularly as per the regulatory frameworks.

Engagement requires a shift in the mind-set of officials in ministries of education, from treating counterpart stakeholder interests as outside concerns that need to be managed, to viewing them as a common public interest that merits dialogue. The implication is that stakeholder views should inform qualification system reform. But before reaching out to external stakeholders, government – whether that means ministries of labour or of education, or agencies for higher education or VET – might explore internally who supports any proposed changes, and identify champions who can promote them. This was clear in the NQF development process in Azerbaijan, which focused first on building a common understanding between ministerial stakeholders before going outside. Therefore, a joint, unified approach to stakeholders could be taken.

Again, to take the development of a national qualifications framework as an example, the ILO points out that “it will be necessary for the government to decide which ministry has the lead role, but also to create an effective internal policy co-ordination mechanism. This could take the form of a joint departmental committee, for example.”21 Moreover, agencies charged with specific reform activities such as establishing an NQF, or improving quality assurance, will need a single direct reporting line to avoid confusion. Even in systems where one ministry has line management (often education) while the agency has a ‘dotted line’ to another (such as labour or employment), confusion over accountability with regard to operational issues can impede progress.

The private sector can appear even more fragmented. This is a consequence of economic change; in most ETF partner countries, micro-businesses and small and medium enterprises, while employing small numbers per business, nevertheless constitute a major – often the greatest – source of employment overall. Smaller organisations face the dilemma of having unmet recruitment needs, and at the same time insufficient resources to do their own training or to engage with education and training providers. Parts of the private sector can be strongly linked to apprenticeships or adult learning, but there are often only weak links to formal education. Even large companies such as multinationals have competing priorities that can overcome well-meaning intentions to participate in VET reforms.

However, there are two main routes that support coordination with private sector actors; direct links with employers on a sector-by-sector basis, and employer representative bodies and federations. These routes are well known in policy circles and can offer important starting points for engagement in both directive and consultative reform processes. A third route is emerging in the area of entrepreneurship education, which offers the potential to act as a bridge between the values and capabilities of education and training providers, the aspirations of graduates, and the needs of employers. Whether people seek to take a job as an employee, or to make a job by creating their own business, entrepreneurship skills and aptitudes are increasingly viewed as essential for successful outcomes. A 2007 study in Ireland found that, from a provider perspective, “entrepreneurship education is an effective means of preparing the graduates for the workplace of the future particularly in the small firm workplace as an employee or as an entrepreneur”, as well as offering benefits to employers and policy makers. Moreover, delivering effective entrepreneurship education will necessitate greater involvement of private sector actors in the design and delivery of curricula, which in turn promotes their involvement in quality assurance, governance, and qualifications reform.

8. Conclusions and recommendations
Qualifications, qualification systems, and qualifications frameworks will be neither sustainable nor credible without the engagement of the range of relevant stakeholders throughout policy discussions, and at both the design and implementation stages. ‘Who’ and ‘how’ we engage are the key questions, and achieving a balance in influence between those involved is crucial. Stakeholder engagement is a demanding, complex, long-term and – frankly – sometimes frustrating process. But successfully building and managing enduring relationships will go a long towards bringing relevance and attractiveness to qualifications, thus serving the interests of all concerned.

Recommendations

• The goal of dialogue is to be productive and ensure that qualifications are understood and trusted by all.
• Map all types of stakeholders and support their involvement.
• Formalise dialogue and support it with appropriate resources.
• Develop a shared communication strategy specifically about qualifications for the audiences represented by stakeholders, and tailor messages for the different groups.
• Be specific in dialogue – don’t waste each other’s time! Dialogue is doing business.
• Accept your share of responsibility for action.
• Remember that providers and end users of qualifications are also stakeholders.

1. Regulating markets of qualifications for lifelong learning
2. Functions and roles in a qualification system
3. The coordination body
4. Institutions regulating for quality of qualifications
5. Deciding between existing or new institutions
6. Combining institutional roles
7. Resource demands in managing a qualification system
8. Conclusions and recommendations

1. Regulating markets of qualifications for lifelong learning
A major challenge for modern qualification systems for lifelong learning is to create systemic links between different types of qualifications. Traditional qualification systems have a strong focus on initial education, divided in educational subsectors; general, vocational and higher education. With the increasing need for lifelong learning a whole range of new qualifications become available for professional development and adult learning. These allow individuals to update and validate their skills and competences to remain employable throughout their working life. We speak about a market of qualifications, offered by old and new providers and awarding bodies, which can be public, private or a combination of both.

The NQF is a tool to bring order in this market of qualifications – so that learners and employers understand the qualifications – but an NQF will not lead to an integrated system of qualifications by itself. This requires coordination between different stakeholders and institutions to set common principles and mechanisms such as a national register of qualifications, common rules on qualification design and assessment and quality assurance. A market of qualifications needs regulation to ensure the quality of qualifications for end users. This is done by external quality assurance mechanisms such as accreditation of qualifications, awarding bodies, assessment centres, providers and study programmes.

Coordination, quality assurance, communication, qualification design and implementation are all processes in a modern qualification system. In this chapter, we look at new institutional arrangements to ensure systemic links between these and other elements of the qualification system – who and which bodies, manage these processes. All countries start from a set of existing arrangements between institutions, but these are likely to change when a qualification system is reformed. Institutional capacities are essential for implementing these reforms and existing arrangements come under scrutiny when countries struggle to implement changes.

We start this chapter with an overview of the functions and institutional roles we consider essential to implement a qualification system. Then we look at two of the main transversal functions: coordination and system review and quality assurance and regulation. We discuss existing and new emerging institutions for development of qualifications and for assessment and certification and we conclude the chapter with a section on implications for resources.

A functional analysis is a good method to analyse existing and required institutional arrangements. The self-assessment tool for this chapter is supplemented by a simple template for a functional analysis that can be used to analyse and discuss existing and required institutional arrangements in your country.

2. Functions and roles in a qualification system

Below, we provide a brief overview of the main functions for implementing modern qualification systems. Annex 3 at the end of the toolkit provides much more details as it specifies the institutional role for each function and provides descriptions of typical executors and concrete examples.

Figure 4. Design, implementation and transversal functions

We have categorised them in two groups: specific design and implementation functions performed by institutions and stakeholders, which have an impact on users and final beneficiaries, and transversal functions (such as coordination and system review, communication, and career information and guidance and quality assurance and regulation) that allow the different aspects to function as part of a system. Annex 3 shows that implementing qualification systems involves different functions and institutional roles. There is no single solution for organising all this work, and arrangements evolve over time. So let us first look at coordination bodies.

Table 2. Functions and institutional roles in implementing modern qualification systems

3. The coordination body
In Chapter 2 we referred to the creation of an ‘’institutional home’’ for the NQF, that should clarify the division of responsibilities between different actors by identifying which bodies are responsible for which functions and institutional roles. This should provide the conditions for the different actors in the qualification system to work together effectively.

Of course, from a sectoral or provider perspective, individual qualifications can be improved one after the other and this can raise their standing in the country. But without coordination it becomes impossible to improve the system and ensure that qualifications share certain minimum characteristics and are systematically compared, linked, and improved. Coordination can have many objectives, including avoiding overlaps and conflicting approaches between stakeholders, agreeing priorities, and strengthening synergies and mutual benefits.

Many of the processes involved in implementing the qualification system are similar before and after the introduction of a NQF. The main differences are not about the processes themselves, but about how these are linked. Before NQFs were established, organisational arrangements rarely worked together as parts of a single system.

Coordination requires the engagement of diverse stakeholders. In Chapter 3, we saw that complementary stakeholder interests and roles can add value to qualifications. But cooperation between stakeholders is not enough in itself to ensure that these gains are available for individuals throughout the education and training system and on the labour market. Dialogue is important, but voluntary approaches by stakeholders cannot achieve systemic change. There needs to be formal institutions to coordinate, and move towards a systemic approach for structural change beyond pilot projects.

Do countries need a coordinating council, a National Qualifications Authority, or both?

Coordination normally starts with a group of stakeholder representatives led by the policy making body, usually the ministry of education. Often there is a division of work between different committees dealing with decision- making and with technical coordination. This works well in the beginning, when the committee has the form of the NQF development working group, as in Azerbaijan and Ukraine. Once the development stage of the NQF has been reached, coordination remains important to ensure effective implementation and cooperation, and monitoring of the qualifications reforms. The critical moment is moving from the committee that conceptualised the NQF to the council that has to support its implementation. In many countries some kind of coordinating councils exist, but few seem to work very efficiently. With a vaguer timeline and diverse tasks, stakeholders tend to delegate these permanent involvements to lower level representatives. Discussions become more technical. Another reason for lack of efficiency is that, with the diversity of stakeholders, councils can be too big to meet regularly. The role of such coordinating councils in influencing government, with no leverage over budgets, can be difficult.

A small executive committee can work well. Turkey has good experience with the five-member Executive Board of the VQA, which collectively takes all major decisions. Under the Turkish Qualifications Framework this model is now duplicated through the TQF Higher Council, which is in charge of decision-making and will be assisted in addressing more technical issues through a wider TQF Council. The efficiency of these bodies is further strengthened by the fact that Turkey has a dedicated Qualifications Authority to support implementation.

A few partner countries have established a Qualifications Authority, and there is a case for doing so. Partner countries that are reforming their qualifications systems with the support of a dedicated institution (Turkey, Kosovo, Georgia) have moved considerably faster than others in implementing reforms. The Authority has to be granted power to act. This implies a clear remit for the new agency that is supported by stakeholders and government. Without such ‘political’ support, existence can be short-lived. In Lithuania, a National Authority of Qualifications established in 2008 to coordinate NQF implementation did not survive a new government one year later. In Georgia, a National Professional Agency was set up in 2007 but also abolished after a year. A new agency was established in 2010 to deal with quality assurance and qualification in VET and HE, this time with wider support.

The credibility of new agencies can be strengthened by making sure they are visible, productive, and able to deliver practical results. They need to communicate with stakeholders and show readiness to support and cooperate with them, demonstrate transparency by providing information on the internet, and have clear reporting procedures to government and stakeholders through their governing bodies. Establishing them at the start of a large donor intervention can be a consolidating factor. Although these agencies are supposed to be independent, stakeholders want control, and therefore prefer public steering or tripartite arrangements. Private initiatives like the National Qualifications Development Agency (NARK) of the Russian Union of Industrialists and Entrepreneurs, or Ukraine’s Institute for Professional Qualifications (IPQ) are welcome to coordinate the contributions by employers, but other stakeholders do not accept them as the formal, coordinators of national reform processes.

The institutional roles of these coordinating agencies are changing with the priorities for developing and implementing the qualifications system. Some countries created a coordinating agency just to establish a new system, not to run it (Australian Qualifications Council, 2009-2014; Malta Qualifications Council, 2007-2012). In most cases, the mandate of the coordinating agency has been reviewed during implementation, sometimes creating completely new organisations. In South Africa, the mandate of the South African Qualifications Authority (SAQA) was reviewed in 2010. SAQA is now sharing its coordinating role with three Quality Councils24, weakening central coordination. In England, the National Council for Vocational Qualifications (NCVQ) was established in 1986 and replaced ten years later by the Qualifications and Curriculum Authority with a wider remit. Then, in 2009, Ofqual was established as an independent regulator for qualifications. In Ireland, the National Qualifications Authority of Ireland merged with the VET regulator (FETEC) and the Higher Education Awards Council (HETEC) into Qualifications and Quality Ireland (QQI), bringing all the qualifications and quality assurance bodies under a single umbrella.

4. Institutions regulating for quality of qualifications
Often, coordinating bodies also have formalized roles in regulating qualifications or quality-assuring the qualifications. The Qualifications Authorities in Turkey, Kosovo and Georgia all have institutional roles in quality assurance. One of these roles is accrediting and registering qualifications in a national qualifications framework. Most countries have adopted at least the principle of a national qualifications framework to bring order to potential chaos, linking the different qualifications (‘old’ and ‘new’, private and public), and adding a sense of logic to the pathways that criss-cross the different subsystems. For qualifications to be registered in a national framework they must meet specific criteria, and the use of registration and accreditation as a quality assurance gateway is strongly recommended. No partner country yet has a fully populated national qualifications framework that includes such criteria. Because qualifications must have currency and remain functional and user friendly, each qualification has an expiration date in the register which varies depending on the need for updating.

Quality assurance of qualifications involves ensuring consistent design and implementation of qualifications, and in particular quality assurance of assessment processes to strengthen trust in the competences of qualification holders. Institutes responsible for accreditation of qualifications and accreditation of providers and assessment centres define national indicators for accreditation.

Quality cannot be imposed top down, but requires a collaborative approach involving different stakeholders. This becomes particularly important if the number of awarding organisations increases. In England and Northern Ireland there are numerous awarding bodies offering general and vocational qualifications. Ofqual accredits the qualifications and accredits awarding bodies, manages the national register, and verifies the activities of the awarding bodies. They in turn verify the assessment activities of providers (or ‘centers’, as they are called in the jargon).

5. Deciding between existing or new institutions

Functions and institutional roles for implementing a qualification system can be performed by existing organisations. There is valuable experience from countries that have established new, specialized, institutions for specific functions and specific institutional roles. The most common are those of the coordinator or qualifications authority, regulators or quality assurance bodies, certificating institutions or awarding bodies, external assessment bodies, and sector skills bodies. Their appearance and mandate differ from country to country. Inevitably, public resources are limited and establishing several specialised bodies may be hard to justify. Some countries continue to rely on existing bodies, particularly ministries of education and schools. Because of conflicting priorities, this can delay implementation. Establishing a new institute can be considered when existing capacities are limited or when conflicting priorities hamper the implementation of new executive responsibilities in existing institutions.

In this section, we explore experiences with new types of institutions for the functions and institutional roles related to development of standards and qualifications, and assessment and certification.

New institutions for development of standards and qualifications

Institutional roles for development and maintenance of standards and qualifications include setting principles and priorities for qualifications development and developing occupational and qualification standards. Professionals who are assigned to work on these tasks need a good insight in qualification needs. This requires anticipating qualification needs, both the current needs and expected changes. This is a continuous process and prerequisite for development and maintenance of relevant qualifications. Information about qualification needs has to come from the economic sectors. Many countries have turned to sector skills councils to ensure that relevant skills are available for the effective functioning and development of their sector. Apart from identifying skills needs, sector skills councils can perform a range of institutional roles, varying from developing occupational standards, developing and reviewing qualifications to assessment of candidates, identification of companies for work-based learning, and funding arrangements.

Sectoral skills councils require the involvement of labour market partners, which is one of the greatest challenges in qualifications development. Many partner countries are establishing sectoral bodies, but these often lack a structural and legal basis and members of sector skills councils are not always representative for their sector. Despite these difficulties, the experience of countries with a longer tradition of sector councils show that involvement of sectoral bodies in development and maintenance of qualifications are still the best guarantee for relevance in vocational qualifications, and for acceptance of qualifications in the labour market. Establishing permanent sectoral bodies with strong involvement of sector representatives takes time but during the process, they can accumulate expertise by learning from their work. And, while sectoral partners may be able to indicate what is needed in the labour market, they do not automatically have the required expertise in learning and assessment. It is therefore important to help them develop this capacity.

In countries with important industrial sectors, such as Russia, Ukraine, and Turkey, the employers themselves have taken the initiative to establish sector skills councils. These councils have strong links with companies and a reform agenda to address their skill needs. They have been particularly interested in developing occupational standards to describe their requirements. In the case of Russia and Ukraine, they established NARK and the IPQ respectively as umbrella organisations to coordinate the work between sectors. Both have worked on developing occupational standards, and aspire to establish independent assessment systems to certify workers against these standards.

In Turkey, the sector skill councils have been integrated under the Vocational Qualifications Authority (VQA). There are different sectoral bodies for the certification of adults against national vocational qualifications that are based on occupational standards. Tripartite sector committees define what kind of occupational standards and qualifications are needed, and the development of standards is undertaken by standard-setting bodies that sign an agreement with the VQA. The process is funded by the sectoral organisation (often employer-driven) that has volunteered to develop the relevant standard. The sector committee reviews the standards before approval by the Authority. The procedure for developing qualifications is very similar.

Permanent sector skills councils can be established on the initiative of government, as in Moldova, where four sector committees have been created and eight more are being developed or planned. Despite regulations for the institutional roles and mandate of these bodies, and support and funding for their work, the sector skills councils in Moldova are not yet operational. Conflicting opinions about operational tasks and responsibilities stand in the way of a clear legal status.

Professional rather than sectoral bodies have been the focus of the professional qualifications system under the Estonian Qualifications Authority, Kutsekoda. This has created easier access to the pool of labour market expertise, but the focus on professions rather than sectors results in a higher number of professional councils, and even more awarding bodies.

In Turkey, Ukraine, Russia, and Estonia, inter-sectoral umbrella organisations support sectoral bodies in performing their tasks. They can also act as clearing houses for inter-sectoral and cross-sectoral competences between the different bodies. The Vocational Qualification Authority in Turkey and Kutsekoda in Estonia, have been established as tripartite organisations, supported by government and social partners. NARK in Russia and IPQ in Ukraine are, as mentioned above, private initiatives.

New institutions for assessing and certification of learning outcomes

Countries are moving gradually away from norm referencing (grading students against their peer groups) to criterion referencing, where minimal standards need to be met. There are different approaches to strengthening trust in assessment, but the most radical is to externalise the assessment altogether and make independent assessors responsible for the assessment. A number of partner countries created special institutions for external assessment. These were often countries with a high interest in higher education but limited places available, requiring a fair selection process.

In Azerbaijan,the State Examination Centre (formerly the State Committee for Admission of Students) has been organising the university admission exam since 1992. Over the years, the State Committee has developed considerable expertise in assessment, which it has shared with the public by producing a magazine and text books to prepare candidates for the exam. It has recently been tasked with organising a similar process for admission to post-secondary VET colleges. The State Committee is independent from the Ministry of Education, reporting directly to the President. The system is considered reliable, but has caused some people to prepare for the entrance exam rather than for completing the full curriculum. That is one reason for extending the committee’s responsibility to the national examinations after lower secondary education and for the completion of full secondary education.

The National Examinations Centre in Slovenia also started from the need to ensure transparent, merit-based, and equitable access to university education. But unlike Azerbaijan, their focus was on technically administering the Matura (upper secondary certificate) as a tool to regulate access, rather than a separate university entrance exam. As in Azerbaijan, it has accumulated assessment expertise that helped it to widen its remit as the technical administrator of the end of primary school and Vocational Matura exams, and an advisory body on exams for Slovenia’s apprenticeship system and adult learning exams. Following accession to the EU, the Centre played an important role in supporting a national committee that assessed the quality of the education system as a whole.

The State Committee for Admission of Students in Azerbaijan as well as the National Examinations Centre in Slovenia perform five of the six institutional roles identified in Table 2:

1. Translate qualification standards in assessment tools.
2. Choose appropriate assessment strategies (including alternatives)
3. Identify assessors and verifiers.
4. Ensure that summative assessment is based on the same standard and is producing comparable results across providers/assessment centres.
5. Ensure that results of assessment are secure and that certification is only issued to successful candidates.

Supporting unsuccessful candidates is not a task of these institutes. Montenegro followed the Slovenian example when it established its Examination Centre, which has also become the key institution for organising the validation of non-formal and informal learning.

The institutions from Azerbaijan, Slovenia and Montenegro in these examples deal with assessment and certification at national level for qualifications that are provided in the formal public education system. These formal qualifications are designed for young people who have yet to enter the labour market. They are not designed for assessing the skills of adults who are already working or looking to change jobs. But new independent assessment and certification bodies are arising and the number is likely to increase because of the increasing need for adult learning.

The VocTest Centres in Turkey assess and certify the skills of adults against national vocational qualifications. Most of these centres have been established by sectoral organisations, and operate as businesses according to current rules. All centres are accredited against the ISO-17024 standard for personnel certification by the national assessment body Türkak, as well as being authorised by the Vocational Qualification Authority (VQA) for each individual qualification for which it issues certificates. At the time of writing, Turkey has 67 VocTest Centres that are authorised for certification of 362 National Vocational Qualifications. All these qualifications have been developed from national occupational standards. To date, the VOCTest Centers have issued 131,002 certificates25. But since certification has become compulsory for, initially, 40 specific ocupations that involve health and safety risks, this number of certificates is growing rapidly.

The VocTest Centres perform these institutional roles:

1. Translate qualification standards in assessment tools.
2. Choose appropriate assessment strategies (including alternatives).
3. Identify assessors and verifiers.
4. Ensure that results of assessment are secure and that certification is only issued to successful candidates.
5. Support unsuccessful candidates.

Ensuring that summative assessment is based on the same standard and is producing comparable results across the VocTest centres is an institutional role of VQA that is currently in development. VQA also develops the occupational standards for the vocational qualifications.

Estonia has been a success story in terms of externalised assessment by independent awarding bodies, formed by professional associations. This small country has 104 awarding bodies that have issued more than 105,000 certificates, the equivalent of nearly seven million certificates in the Turkish context. Employers and ministries of labour in Russia, Kazakhstan, Ukraine, and Azerbaijan are aiming to establish similar systems in their countries and have, like Turkey and Estonia, started with the development of occupational standards.

Independent assessment centres can strengthen the confidence in qualifications, but the costs of establishing seperate centres should not be underestimated. It can take a long time before systems are fully up and running, and considering the poorly equipped state of training centres in many countries, one might question whether the priority really is externalising assessment. There are other ways to make assessment more independent, through better internal quality assurance processes and effective oversight; by involving stakeholders from the world of work in assessment; by training and accrediting professionals to act as assessors; or by encouraging assessment of vocational skills in authentic settings, such as enterprises, and by sharing the responsibility.

New awarding bodies

Many NQFs are established to support the recognition of lifelong learning in name, but in practice only cover the formal education system; widening the pool of qualifications in the NQF can help increase opportunities for recognising lifelong learning. Bringing qualifications into the NQF that are not the product of established approaches used by the ministry of education, or providers under the ministry’s control, can help in rethinking and modernising qualifications. This has the effect of strengthening links with the labour market and improving confidence in qualifications not issued by ministries of education.

England has a tradition of private awarding organisations developing, quality assuring, and issuing their own qualifications for general and vocational education, including higher level qualifications. These organisations have to fulfil certain requirements before they can be recognised. Only recognised organisations can develop qualifications that can be accredited in the NQF or, better, the register of accredited qualifications. There are currently 210 recognised awarding organisations dealing with vocational and general qualifications

Sectoral bodies in France develop their own qualifications as part of the NQF, a process that has begun in Belgium. The establishment of sectoral frameworks will probably lead to an increasing number of sectoral awarding bodies in partner countries, but we do not have any specific record yet. A recent study in Ukraine25 showed a wide variety of qualifications and awarding bodies that could be considered, including some with very high standing in the labour market. The Qualifications Authority in Kosovo is already accrediting providers as awarding bodies. The same is done by the Department of Skills Development in Malaysia, which regulates the award of Malaysian Skills certificates based on National Occupational Skills Standards that are part of the Malaysian Qualifications framework. Similar systems exist in Singapore and Timor-Leste, where qualifications frameworks have been established.

6. Combining institutional roles
An ETF study of 18 existing bodies shows that most specialised bodies supporting the implementation of qualifications reforms perform a mixture of institutional roles. In reality, the dividing lines between sectoral bodies, awarding bodies, independent assessment bodies, coordinating institutions, regulators, and quality assurance bodies are less clear-cut. Many institutions seem to support both the quality of provision and the quality of qualifications. We see clear synergies in many cases in bringing different executive functions under one umbrella. It can help to speed up the impact of new qualifications on curricula and programmes, but there are also risks in combining institutional roles that can jeopardise the transparency of verification processes.

Fragmentation of functions between institutions that work in silos is, however, worse. It can create competing institutions and produce considerable inefficiencies and delays in implementing reforms. Many examples of such competing approaches can be found between quality assurance bodies and qualifications bodies for HE, VET, adult learning and general education, between ministries of labour and education, between VET and HE, and between central bodies and regional centres of excellence. Despite market-driven solutions that are deemed more efficient, frequently being attributed to competing models, in practice cooperation and consensus are more efficient.

Under the influence of the Bologna Process and the European Standards and Guidelines, quality assurance agencies have been established in many partner countries to monitor universities’ quality assurance of provision and awarding processes. Some countries have taken the opportunity to convert these new bodies into their qualifications and quality assurance agencies for lifelong learning. Good examples are QQI in Ireland, and the National Centre for Education Quality Enhancement in Georgia. ANACIP, the new Quality Assurance Agency for Higher and Vocational Training in Moldova could also develop in that direction, but it will depend on how the ministerial restructuring will end.

7. Resource demands in managing a qualification system
All processes related to qualification development require capacities and resources, and when looked at purely from a cost perspective, no change seems cheaper than reform. In most EU countries, these capacities have been developed gradually as systems evolved. In ETF partner countries, existing capacities are a severely limiting factor, considering that – almost without exception – a radically different concept of qualification is proposed, and all qualifications are therefore in need of revision. New capacities often depend on donor funded projects and become a limiting factor when the results of such projects, or new policies and legislation, must be implemented nation-wide.

The unit costs per qualified person can be determined by a series of factors. These include:

1. Preparing and maintaining relevant qualifications.
2. Translating these into training programmes, and organising an appropriate learning experience (which requires curriculum development activities, teacher preparation and retraining, making learning tools and learning environments available) that are focused on an active role for learners, including exploring alternative options.
3. Making career advisors and employment services aware of new opportunities.
4. Informing potential learners and their families about such opportunities.
5. Informing potential employers and learning providers about new qualifications.
6. Recruiting learners and/or candidates and evaluating them before they begin their programmes.
7. Training, assessing, and certificating learners.
8. Monitoring learning and assessment.
9. Quality assuring learning, assessment, and certification.
10. Monitoring graduates.

These costs depend to a large extent on the number of learners per qualification. The larger the variety and number of qualifications in the framework the higher the costs of development, verification and approval, and of translating them into learning and assessment routes. The larger the number of pathways and actors (assessment centres, awarding bodies, sector committees, QA bodies) the higher the costs for ensuring coherence across the framework. More organisations also means more overheads. There must be an expectation that each reform measure, each new qualification, will make an impact, before changes are made. This is difficult, as systems are often untested.

Benefits

The benefits need to be considered carefully before costs are calculated. Benefits are evident for individuals, for employers in terms of productivity, and for society in terms of better access to the labour market, less time for induction, improvements in social demography and career developments, better remuneration and purchasing power, social inclusion, adaptability, and active citizenship. The key criterion for defining the size and remit of supporting institutions, and the appropriate numbers of qualifications and programmes on offer, is linked to an analysis of the expected impact and benefits of the new qualifications. Appropriate qualifications are needed to recognise lifelong learning, and to allow for alternative pathways to achieving them. More involvement of stakeholders means greater relevance, which drives what people can do with their qualifications.

The literature speaks of wage returns, employment returns, and spill-over effects, such as more motivated and healthier individuals, more civic engagement and positive attitudes towards society, and even intergenerational effects, stimulating younger people to learn. Many of these benefits only appear over time, and are difficult to assess in advance. But there are studies which looked at the benefits once systems are up and running. They show that certain abrupt reforms were miscalculated, while others that were widely discussed with stakeholders were more successful.26 It is important to determine from the outset what kind of impact is desirable and achievable with qualification reforms, in order to establish how best to obtain these results and make sure that goals are widely shared among all stakeholders.

Who pays?

Cost sharing can make qualification reforms more affordable, and there are three main sources – government, employers, and individuals and their families. In deciding who should pay, there are different principles about who gains more from a new qualification. This depends on the individual and on social returns. Governments should be aware of the fact that public investment in a qualified workforce is needed if effects are expected across the population as a whole. However, both individuals and companies may be ready to contribute, and there are many incentive schemes that can be used to spread the burden.

8. Conclusions and recommendations
There are contextual factors that determine what works best in a country, such as its size and complexity of its system. Existing institutions can barely cope with all the changes implied in reforming qualifications, meaning that progress is slow. New institutions are needed to accelerate reform. There are no perfect answers, and no one-size-fits-all solutions. But implementation is faster where ministries can focus on policy, and executive functions are delegated to agencies. New types of agencies are being created and different executive roles are often combined, which has the advantage of greater coherence between functions, and potentially reducing costs. However, putting too many tasks under one roof can blur responsibilities between the agency and other actors; for example, if an agency’s assisting or facilitating work is mixed with prescribing how others work.

Therefore, concentrating all related tasks in a single technical agency is not a feasible solution. Concerted efforts are needed and professionals dealing with the implementation of the qualification system must have appropriate time and resources to do their job. Taking costs and benefits into account, reviewing and, where necessary, amending structures and roles, finding the balance between directing and delegating – these are all part of the task of agreeing institutional roles.

Recommendations

• Review existing institutions’ capacities and identify gaps and overlaps. Consider creating new institutions to accelerate reform.
• All functions required for the implementation of systemic change must be located in specific institutions.
• Don’t work in silos, but do ensure a clear division of mandates and tasks between institutions, avoiding conflicts of interest.
• Manage competing remits between different ministries.
• Designate a coordinating institution, for instance through the creation of a new institution.
• Professionalise at every level, because voluntary processes alone will not provide sustainable results.
• Communicate, coordinate, and quality assure continuously.
• Consider combining functions and roles under a single institution.
• Look for affordable, fit-for-purpose solutions.
• Regularly review institutional mandates to avoid rigidity in processes, and to adapt to changing circumstances.

1. Quality assurance for trust
2. Quality assurance in qualification standards, assessment and certification
3. Quality qualifications – concepts and characteristics
4. Building the qualifications quality chain
5. Conclusions and recommendations

1. Quality assurance for trust
We have recommended that countries design proportionate legislation, establish inclusive dialogue with stakeholders and clearly allocate roles to institutions. These three elements contribute to relevant and trusted qualifications. Legislation establishes the basic principles and minimal requirements for quality assurance and defines the roles of the actors in the quality assurance system, including the roles of regulatory bodies and those directly responsible for delivering qualifications through learning and assessment. Quality assurance and quality enhancement requires active feedback and cooperation from stakeholders. Dedicated institutions are responsible for safeguarding standards across the system as a whole, and for ensuring equal access, fairness and impartiality for learners, wherever they are located. However, quality assurance is particulary critical at the level where qualifications are actually delivered. A fourth element is therefore necessary to ensure quality in the final outputs, the qualifications and the qualified individual. This is the quality assurance of qualifications. The quality assurance of qualifications links these three other elements as criteria are prescribed in legislation, stakeholders bring relevance, and institutions perform quality assurance functions.

In Making Better Vocational Qualifications27, we recommended that, in order to compensate for the lack of trust in qualifications, ETF partner countries need to put in place quality assurance systems that are robust, up-to-date, and fit-for-purpose. Here, we will further illustrate, develop, and scrutinise this recommendation.

For this purpose, we need to clarify a few concepts and impose a few limitations, as quality assurance of qualifications can encompass a vast number of processes and functions. Drawing a distinction between quality assurance in qualification systems and wider quality assurance is not easy, although the ultimate aim of QA is the same: trusted and quality qualifications. But, to simplify, our concern here is with standards, assessment and certification.

We will start by saying something about quality assurance in general, briefly addressing related concepts such as quality culture, quality management, and the quality cycle. After that, we will elaborate on the ETF approach to quality assurance in VET, which is one of the foundations from which the recommendations in this tool-kit will be drawn. That will lead us to the heart of our concern in this chapter, something we have called the ‘qualifications quality chain’. As noted, quality assurance involves many factors, and there isn’t the space here to be comprehensive. Therefore, our aim is to dig deeper into what we see as the most important functions and processes from a qualifications perspective; qualification standards, assessment, and certification.

As we have seen in earlier chapters, qualifications standards define the requirements for the award of qualifications. The EQF definition of qualifications, presupposes that qualifications are awarded by competent bodies after establishing that learners have demonstrated to have met the intended learning outcomes that are defined in the standards, through an assessment and validation process. However, the EQF definition cannot be generally applied to all existing qualification standards, but is rather an orientation for good standards. There are different types of standards that set the conditions for issuing qualifications, and the term qualification standard is used only in exceptional cases. We therefore speak about the standards behind qualifications that are further discussed below.

Assessment is the process of verifying whether someone meets the learning outcomes. We are in particular interested in assessment processes that count for deciding whether someone can get the qualification, the summative assessment. This type of assessment can take place at the end of the learning process, but not necessarily. It is important that assessment is impartial and objective and the right things are assessed in the right way. Different methods can be combined as we will see below.

Certification is the process of actually awarding the qualification to a person. It should be performed by an institution that has the right to issue qualifications, after double-checking that the person has indeed met all the necessary requirements.

Before taking a look at current approaches for quality assurance, two starting points for this chapter need to be made explicit. First, a qualification is a qualification. The main focus throughout this tool-kit is what are often referred to as VET qualifications or qualifications with a strong emphasis on labour market relevance, but distinctions between general, VET and higher education qualifications are increasingly blurred. So much of what we advocate is applicable to various types of qualification. Second, the focus of the chapter is quality assurance and not quality per se. With that said, it is impossible to say anything on quality assurance without saying something on quality. Therefore the chapter will also address the quality of qualifications, although not extensively, but with a proposal for minimum quality criteria. What is important to keep in mind is that quality assurance alone does not guarantee quality; what is meant by ‘good’ quality needs to be defined and broadly agreed within each context where it is to be assured.

2. Quality assurance in qualification standards, assessment and certification
Generally, and not only in ETF partner countries, the spotlight has been on quality assurance in VET providers, often through accreditation, and on external evaluation conducted by school inspectorates or bodies with similar functions. The development and implementation of national qualifications frameworks has been a means to review quality assurance arrangements and mechanisms, and an initiation point for change and reform. For many countries, this has occurred in combination with reforms driven by the Bologna Process for higher education. Changed regulations and new institutions are being introduced, and quality assurance is high on the agenda. Therefore, there is an ongoing shift away from emphasising what can be called input factors, towards outputs; that is, a focus on what learners can do with their qualifications. At the moment there are still more questions than answers. For instance, we still need to know which factors make qualifications and the process of awarding them more trustworthy than others. In countries where there is high trust in VET systems (and where participation in VET is high), there may be no need for explicit quality assurance measures. These systems can afford to be ‘lighter’ in terms of quality assurance, making them more proportional in terms of results from inputs. However, for our partner countries there is no copy-and-paste solution, as systems with high trust have evolved over time in contexts which were, and remain, different.

In our partner countries, the dominant approach to quality assurance seems to be a centralised version of the so- called prescriptive model that focused on control rather than empowering providers28. One national body designs and specifies assessment methods, and assessment is centralised or delegated to providers in name of the central authority using the centrally established exams. In addition, the national body is often in charge of quality assurance, validation,29 and awarding of the certificate. This approach goes hand-in-hand with centralised governance of VET and VET qualifications, and quality assurance based on the compliance of providers, mostly public VET schools, with prescribed rules and regulations combined with centrally organised inspection and audit.

However, this approach tends to be of limited efficacy in relation to the labour market relevance of VET programmes and the currency of qualifications, and often fails to support VET quality improvement or meet the expectations of learners, employers, and funding bodies. Whilst the relevance of standards or the effectiveness of their application and evaluation may be under question, certain quality assurance measures which aim to ensure relevance for changing needs remain valid, albeit in need of modernisation.

Countries are experiencing an increase in number and range of programmes and qualifications, offered by NGOs and private providers and sometimes mixed private-public bodies. There are also growing numbers of occupational standards available, often donor or project-created. Countries are responding by shifting to more external assurance, extending QA functions to e.g. VET agencies and seeking to ensure a vital element of quality – relevance – through enhanced involvement of employers and labour market actors, for example through sector skills councils. Examples of countries increasingly looking at external quality assurance, via VET agencies or quality assurance bodies to accommodate this change can be found. In 2010, Georgia established its National Centre for Education Quality Enhancement, which deals with all sub-sectors of education and training; and the NQF coordination as well.

Moreover, the use of labour market and skills demand data, both in terms of collection and analysis (both quantitative and qualitative), is still a challenge in practice. Using better data and information on occupations and skills needs; and engaging more systematically the stakeholders – when developing the standards behind qualifications. An example is the Competence-Based Approach in Morocco, and its new initiative of Sector Observatories for skills.

There seems to be greater attention paid to quality assurance from outside the traditional centralised and inspection-focused approaches. This is linked to a new emphasis on the processes of developing the standards behind a qualification, in terms of how and by whom it is done. But it still appears that the process of awarding qualifications, including both assessment and certification, is neglected.

The findings of a recent Cedefop research study on the quality of certification in Initial vocational education suggest that this is also the case for EU member states. 31Countries represented in the study do not define the certification process; however, elements of certification (assessment, verification and grading, and awarding) were found to be present in all the countries studied. The process of certification was defined as “the multiple (and sometimes overlapping) processes of assessment and verification of learning that lead to the awarding of a qualifications or part thereof. The ultimate goal of a certification process is to ensure that the learner has acquired the required learning outcomes (knowledge, skills and competence) which is then certified by the awarding of a qualification”. In some countries qualifications were awarded on the basis of a final assessment at the end of the studies, in others through a cumulative process of modularised (or unitised) assessment of parts of the qualification. The study showed a wide variation of practices among and within member states on how assessment, verification, certification and quality assurance were implemented, lacking a common set of principles.

The central message of the study appears to be that, “to strengthen trust in certification, results across the system based on the same qualification standards must be comparable. Comparability of results ensures that holders of the same qualification have actually achieved the learning outcomes required for it and therefore qualifications can be trusted”33. Another important finding is that only few of the countries studied explicitly address certification and its links with qualifications as an essential aspect of quality assurance policies. More integrated and comprehensive approaches are needed.

The study concludes with eight recommendations, which are valid for ETF partner countries to consider:

1. Clearly articulate certification in VET policies.
2. Define and use learning-outcome based standards appropriately
3. Strengthen involvement of labour market stakeholders in certification and relevant quality assurance processes.
4. Support a common understanding of certification requirements among stakeholders.
5. Ensure that assessors are competent and trained.
6. Share responsibility for quality assurance of certification at all levels.
7. Strengthen evaluation and review of certification.
8. Consider the possibility of a handbook to apply quality assurance principles (in this case EQAVET34) in a coherent and holistic way.

Table 3 summarises characteristics of QA of providers compared to QA of awarding qualifications. Countries need to quality assure both providers and the awarding process but countries need to give more attention than they do now to quality-assuring the awarding process.

Table 3. Comparing quality assurance approaches

3. Quality qualifications – concepts and characteristics

Definitions of quality assurance differ to some extent between different contexts, and are usually connected with concepts of quality control and quality management. To achieve trust, the sum must be greater than simply putting the parts together in a mechanical, ‘tick-box’ fashion. The concept of fostering a quality culture is vital.

A quality culture deliberately aims at reflecting on all performance and gathering feedback and information to enhance quality continuously. It is based on shared values, beliefs, expectations, and commitment to quality and at the same time contains an element of planned processes aimed at enhancing quality36. This is often referred to as ‘continuous improvement’ and should be the objective for any quality assurance system. Explicit feedback mechanisms, undertaking self-assessment, willingness to learn from mistakes, and going through external evaluation for the sake of improvement are some examples of methods for underpinning a quality culture. A concrete example could be that qualifications need regular review and updating. The commitment of all actors to this, as a natural part of a qualifications system, suggests a view of quality as more than just a control mechanism.

Any quality assurance system needs to find the right balance between quality control and quality improvement. Systems that are too rigid tend to focus more on control and less on improvement. Fostering a quality culture by means of quality management, understood as the activities used by organisations to direct, control and co-ordinate quality37, including formulating a quality policy and setting quality objectives, is important for all actors within a qualification system. Quality management should encompass, in parallel to control, quality planning, assurance and improvement.

The quality cycle, most commonly derived from the PDCA management tool, is used for control and continuous improvement of both products and processes38. PDCA stands for Plan-Do-Check-Act (or Adjust), and creates a feedback loop that supports quality improvement and the establishment of a quality culture. The European Quality Assurance Reference Framework for VET (EQARF) provides a resource for countries reforming their qualification systems that builds on the PDCA cycle. The aim of EQARF is to improve the quality of VET assurance, and increase the transparency and portability of qualifications; It supports countries to document, develop, monitor, and improve quality management.

Stage 1 is about setting up clear, appropriate and measureable goals and objectives in terms of policies, procedures, tasks and human resources.

Stage 2 is about establishing procedures to ensure the achievement of goals and objective (e.g. development of partnerships, involvement of stakeholders, allocation of resources and organisational / operational procedures).

Stage 3 is about designing mechanisms for the evaluation of achievements and outcomes by collecting and processing data in order to make informed assessments.

Stage 4 is about developing procedures in order to achieve the targeted outcomes and / or new objective. After processing feedback, key stakeholders conduct discussion and analysis in order to devise procedures for change.

EQARF also encompasses a set of 10 indicators, covering such factors as the number of providers applying internal quality assurance systems, participation and completion rates for VET, or funds invested in teacher CPD. The EQARF indicators do not explicitly address assessment and certification. The common principles for quality assurance that are annexed to the EQF recommendation40 focus to a large extent on providers and programmes with no explicit reference to the process of awarding qualifications. At the same time, it states that the emphasis should be on outputs and learning outcomes. The principles of both EQAVET/EQARF and the EQF recommendation should be fully respected, while being translated into more practical criteria and requirements.

Figure 4. European Quality Assurance Reference Framework – Quality Cycle

A recent ETF working paper defines quality assurance in VET as “The composite measures established to verify that processes and procedures are in place, which, when effective, ensure the quality and quality improvement of VET. The measures often have a regulatory or legislative underpinning and status. The measures relate to quality standards with underlying principles, criteria and indicators.”41 The same working paper states that the overall purpose of quality assurance is to support the “attainment and maintenance of VET quality standards”, and that the objectives are to “support the provision of high quality VET and the attainment of relevant qualifications”.

As stated above, the aim of the ETF approach is of course to ensure good VET, which in this context consists of five key features, through which it:

• Enables access to decent jobs and sustainable employment
• Fosters capabilities that enable progression and further learning
• Is attractive, inclusive and accessible
• Is responsive to labour market, societal and individual needs
• Leads to nationally, or even internationally, recognised qualifications or credentials.

These are also relevant from a quality qualifications point of view. Furthermore, Table 4 shows the five main related elements which should be included in a systemic quality assurance approach.

From these elements, we focus on standards and at assessment, validation, and certification. For a review of quality assurance from a broader perspective, the above- mentioned ETF working paper is a good place to start.

For each of the five main areas in Table 4 there are quality assurance criteria, and for the two areas that we are focusing on the criteria are specified as follows:

Qualifications standards:

•   Based on research and reliable evidence defining specific skill needs
•   Developed by the state and social partners
•   Monitored and reviewed regularly
•   Used to underpin verified programmes, curricula and contents.

Assessment, validation and certification:

•   Based on standards
•   Managed and executed by appropriate stakeholders
•   Recognised by employers
•   Supported by appropriate, objective, and reliable mechanisms.

Defining quality is always difficult, because definitions frequently fail to be both appropriately extensive and sufficiently precise. Based on our experience from partner countries, as well as from other countries with emerging qualification system reforms (often founded on introducing qualification frameworks as a way to enhance quality), our approach is to find the common denominators, or as we call them minimum criteria for the quality of qualifications, as shown in Figure 6.

Table 4. Main elements of a systemic quality assurance approach

This should be seen as a work in progress and as a starting point for defining quality in respective contexts. The focus of the criteria is very much on VET qualifications and relevance for the labour market and we need to keep in mind that quality and relevance are linked to the purpose of the qualification, and that is not always employment or labour market needs.

Figure 6. Minimum criteria for the quality of qualifications.

The proposed criteria for each of the five areas are as follows:

1. Relevance for the labour market and individuals

• Involvement of labour market actors in defining needs for a qualification
• Justification that the sector and the occupation has a relevance for a country.

2. Standards behind a qualification

• All standards are (learning) outcome-based
• All standards behind a (VET) qualification should relate to skills and competence requirements for an occupation or a group of occupations
• Involvement of labour market actors in defining the standards of a qualification.

3. The learning process

• The learning outcomes relate to the standards behind a qualification (skills and competence requirements for an occupation or a group of occupations)
• The learning process has a substantial practical component for students to acquire skills and competences and not only theoretical knowledge.

4. Assessment for certification

• Assessment is based on the standards behind a qualification (skills and competence requirements for an occupation or a group of occupations)
• Assessment has a substantial practical component for students to acquire skills and competences and not only theoretical knowledge
• Trained professionals are involved in assessment of skills and competences.

5. Certification

• A numerical level is allocated to the qualification in order to compare the level with related qualifications from other countries
• The certification has national value and is awarded by a competent body (ministry, federation, etc.)
• The qualification allows for progression to further education or training.

Some partner countries are using variations of the “quality cycle” – plan-do-check-act – that encourages a feedback loop, supports quality improvement and the establishment of a quality culture. This can be illustrated by the on-going review of the NQF in Georgia, using stakeholders’ feedback after a few years of implementation.

4. Building the qualifications quality chain
Quality assurance for quality qualifications consists of two broad processes :ensuring that qualifications are relevant and have value; and that the people who are certificated meet the conditions of the qualification. The processes, or steps, in the ‘qualifications quality chain’ target these aspects more closely. To guide partner countries when building their legal and institutional infrastructure, quality assurance should not be addressed separately, but should be an integrated part of developing or reforming that infrastructure, so as not to risk it becoming an afterthought.

Figure 7. The qualifications quality chain

Here it is helpful to use a technique called back-chaining. Normally the chain would start with developing the standards, or even with developing the levels and descriptors of a qualifications framework. We have chosen to start from the intended ultimate outcome, so that the other stages or processes can be identified and planned to contribute consecutively to achieving that goal. When developing quality assurance policies and putting quality assurance measures in place, there is a risk of ending up with an infrastructure that is not fit for purpose, and that relies too much on what already exists. To implement new and, hopefully, more efficient solutions it is sometimes an advantage to start from the end.

Even if we were only discussing certification, assessment, and standards, quality assurance of the system as a whole also plays an important role in building trust and quality, as well as reviewing the functionality of the arrangements regularly. When developing quality assurance focused on qualifications there is no one-size-fits-all solution, but a few key elements need to be taken into consideration.

Certification

Often, the term ‘awarding of qualifications’ is used, encompassing both the process of assessing learning and the issuing of a certificate demonstrating that the learner meets the specified learning outcomes. Normally – at least, in the EU – certification is defined as including assessment, verification and grading, and awarding. Verification and grading is the process that follows assessment and is the part of assessment where the result of that assessment is verified against the relevant standard. Awarding of a qualification should be “understood as issuing a certificate that officially attests that an individual has achieved the identified learning outcomes”. This is the final step in the process of an individual attaining a qualification. However, here we are separating the awarding of the qualification from the assessment step in order to make more detailed recommendations in relation to quality assurance.

Certification can be towards a full or partial qualification (the latter more often as a result of a process of validating non- formal and informal learning). The certificate itself can be a diploma issued after the completion of a learning or training programme, it can be in the form of a licence to practice a specific profession (e.g. for regulated professions in health care and medicine), or it can be a certificate that is specific to a company or organisation and part of their personnel training scheme (e.g. Microsoft or Cisco).

Certification can be done in different ways, the difference mainly depending on who has the authority to issue the certificate. In public education systems, for what we normally refer to as formal qualifications, it can be done in either a centralised or de-centralised manner; centralised meaning issued by a national body (in many partner countries the ministry of education), and de-centralised meaning by the training (or assessment) provider. For vocational qualifications, whether formal or non-formal, there is also the possibility of qualifications being issued by a specially appointed body that isn’t a fully centralised body and that may or may not also be a training provider.

Qualification systems and frameworks usually encompass all three ways of issuing certificates, and the difference is mainly due to different types of qualifications within the system or framework. For that reason, it is not feasible to have one quality assurance system in place covering all these variations, but rather QA needs to be aligned with whoever is authorised to issue certificates. And that authorisation need to be an explicit part of the system itself. This is why a set of overarching quality assurance policies, principles, and criteria is more important than a detailed quality control system. The authorisation will, in most systems (countries), need to be regulated by law – in some cases this is what the NQF law aims at achieving – and a designated institutional infrastructure needs to be in place. In our experience, the authorised issuer of certificates often follows an established legal and institutional structure instead of the opposite, structures being set up to be fit for purpose, with the aim of achieving good quality qualifications that are relevant and trusted.

From a quality assurance point of view, the relevant, open, questions are:

• Who should issue the certificates and how are the issuers appointed?
• Is there a need for a regulator to oversee the awarding bodies to guarantee quality?

The distinction between the assessing institution and the ‘competent body’ that actually issues the qualification can be an important one. The trust in the organisation that issues the certificate or externally validates the awarding process strengthens the reputation of the qualification. In the UK there is a clear distinction between the awarding body in VET that issues a branded qualification (e.g. a BTEC qualification issued by Pearson) and the provider (or centre) that assesses the qualification.

In Turkey’s National Vocational Qualifications System, under the Vocational Qualifications Authority, qualifications are part of a national register and based on national occupational standards, but they are issued by the authorised certification body or VOCTEST Centre that has carried out the assessment. This is related to the requirements of the ISO standard 17024 that does not allow accredited assessment bodies to sub-contract the certification to VQA. The result is that for the same national vocational qualification, eight different authorised certification bodies issue qualifications. To ensure that those qualifications are still recognisable as national qualifications, all certificates have the same format and include the logo of the authorised certification body (VOCTEST Centre), the Vocational Qualifications Authority (MYK in Turkish), and Turkak (the accrediting institution for ISO-17024).

Figure 8. Example certificate

In Bosnia and Herzegovina the Baseline Qualifications Framework is adopted but not operational and there is no register of quality assured qualifications yet. Seeking recognition beyond the country’s borders, the Faculty of Electrical Engineering obtained international accreditation for its study programmes by ASIIN, a German specialised IT accreditation agency, while Mostar University supports its IT students in acquiring MikroTik, CISCO, and Microsoft-certificates.

Professional bodies have an interest in regulating access to their profession and function in many cases as awarding bodies. In Estonia 104 professional bodies are issuing professional qualifications under Kutsekoda, the Estonian Qualifications Authority. In Ukraine, a recent ETF study shows that there is scope for a more active role of professional bodies to act as awarding bodies in the NQF, building on what these professional bodies already have been doing over recent years.

Assessment

Assessment is changing. One of the main reasons is the shift towards competence-based qualifications and curricula, and towards learning outcomes. There is still a prevalence of systems that favour final instead of modular assessment, teachers being assessors, and assessment mostly taking place in schools and focusing on theoretical tests.

With the development and ongoing implementation of qualifications frameworks and learning outcomes, assessment becomes the necessary link between the individual learner and the content of a qualification. As such, it is a crucial process to enhance trust in qualifications. Assessment is most commonly defined as the process “of identifying the extent to which a learner has attained particular knowledge, skills and competences”.50 Assessment can relate to both the whole qualification or to parts. In the revised VNFIL guidelines of 2016, assessment, in the context of being one of four phases of a validation process, is defined as “the stage in which an individual’s learning outcomes are compared against specific reference points and/or standards”.51 In Making Better Vocational Qualifications we express the same understanding of assessment slightly differently: “To be awarded a qualification based on learning outcomes, an individual needs to demonstrate competence against a relevant qualification standard.”

The core of these different ways of defining assessment becomes clear when seen as a process in which there needs to be an agreed yardstick, and which takes place after the completion of learning (usually referred to as summative assessment). Some further questions arise from the perspective of quality assurance and enhancing trust:

• Who should carry out final (summative) assessment? Should it be externalised, as in de-coupled from provision or provider? How should assessment bodies be authorised, and by who? Does it always have to be final, or can assessment be done for parts of qualifications?
• Who should do the assessment itself – who are the assessors? Should there be a formal requirement for assessors, and if so how should they be appointed? Should assessment be done in teams?
• How can principles such as validity, reliability, objectiveness, fairness, and fit-for-purpose be ensured in assessment? Can the same principles be applied to quality assurance of VNFIL and summative assessment in formal education?
• How, and by who, are assessment standards developed? How should alignment with these standards be assured? How are they linked with the qualifications standard?

We are not offering prescriptive answers here, but the natural conclusion is that enhanced attention to assessment, its standards, procedures, and professionals, is necessary.

The Examination Centre in Montenegro is the national expertise centre and the external QA body for examinations in the country. It also deals with organising external and national assessments. It prepares, organizes, and conducts examinations and provides advice to institutions and individuals. It trains examiners for the assessment of National Vocational Qualifications and is in charge of Montenegrin language exams for people who seek citizenship. The National Assessment and Examinations Centre (NAEC) in Georgia and the State Student Admission Commission in Azerbaijan are similar bodies that are gradually expanding their remit towards the areas covered by the Montenegro Examination Centre.

The Federation of Employers in Ukraine is developing a new system for professional qualifications that are based on occupational standards. With the support of sectoral committees it has developed 46 occupational standards, and is advocating the establishment of independent assessment bodies like in Estonia and Turkey. The draft Law on Education in Ukraine, which establishes the basic principles for lifelong learning, is integrating the independent bodies for the assessment of professional qualifications. The Federation of Employers wants to promote the role of competent assessors who practice the occupation rather than schoolteachers. This is very similar to the systems that were developed in Estonia and Turkey during the past decade.

The handbook underpinning the NQF in Kosovo provides a detailed description of how assessment should be implemented under the NQF53. The handbook looks at the practical aspects of the assessment after defining the legal basis for assessment in the NQF law and explaining the underlying principles, stating that the assessment and awarding processes must be fair and objective, flexible, valid, reliable, sufficient, practicable and cost-effective, and transparent. In Kosovo, assessment is decentralised to providers and assessment centres that are accredited for assessment and certification. A central role is given to competent assessors. Assessment should be carefully planned and take into accounts the needs of candidates. The process and methods of assessment are discussed in view of how sufficient evidence can be collected to demonstrate that learners or candidates have obtained learning outcomes, combining different assessment methods.

Learning Pathways

Learning pathways can differ between different qualifications, between learners, and of course also in time. The process of learning is what links the developing with the awarding of qualifications. In terms of quality assurance, we are treating this as a ‘black box’ and will stop at encouraging countries to include the possibility of individual learners in their qualification system attaining qualifications through different pathways, through validation of non-formal and informal learning, and through the possibility of having studies and work experience from abroad recognised. These pathways might include, among others, different types of provision such as VET programmes in VET schools, distance learning courses, open educational resources, and so on.

• What types of learning pathways towards a qualification are taken into account at national level?
• Are all learning pathways taken into account in quality assurance policies and measures?

Placing qualifications in a framework

If a framework of qualifications exists, the process of including individual qualifications in the register of qualifications of that framework is one of the most important quality processes. It gives the opportunity for benchmarking nationally and often internationally, if the framework is (or will be) referenced to the EQF. Comparison of content (learning outcomes) and levels can be used as a quality check. Qualifications that are allowed onto the framework must meet prior set-up criteria for being added. The criteria and process may differ between different types of qualifications, the level at which formal qualifications are placed often being decided by national authorities. Those frameworks that are open to non- formal qualifications (i.e. qualifications that are not part of the formal education provision) usually have a special procedure for adding them. In both cases, the principle of best fit should be followed. In particular, for non-formal qualifications, this process must function as a gate-keeper in terms of quality, and only qualifications that meet requirements should be placed in the framework. The Twinning Project on Vocational Education in Ukraine developed a methodology of placing qualifications in the NQF that have not been fully brought in line with the NQF yet, based on a three-step approach: comparison, consultation, and decision-making.

The individual qualification is of a certain qualification type that has been indicatively placed in the NQF. This is a starting point but does not predetermine at which level the qualification should be placed. The learning outcomes of the individual qualification are mapped in line with the four domains (Knowledge, Skills, Communication, Autonomy and Responsibility) of the NQF and compared to the NQF level descriptors.

Current state education standards often lack a coherent description of the learning outcomes and it is therefore proposed to use either the curriculum or the occupational standard as the source of information. On the basis of a comparison between the learning outcomes of the qualification and the level descriptors per domain a match is established. The match may not coincide fully with the expected level descriptors and it is therefore proposed to allocate the learning outcomes to the closest matching level.

The arguments are brought together for allocating a NQF level and recommendations for reformulation of the qualification are also formulated. In order to strengthen transparency and increase trust in the allocation of levels, appropriate stakeholders from the world of work and education should be involved at the level of making the comparison between learning outcomes, through consultation of the results with a wider group and finally in decision-making. The advantage of this method is that levels can be allocated to similar qualification in analogy, even if not all of them have been described in the appropriate format in learning outcomes.

In order to support the evidence base for referencing of the NQF to the EQF in the former Yugoslav Republic of Macedonia, a comprehensive inventory has been made of all existing qualifications that could be placed in the NQF through a transparent process. The inventory encompassed educational qualifications, including two-, three-, and four-year VET programmes and professional qualifications, including adult education programmes, civil aviation programmes and the master of crafts exam. Besides the comprehensive inventory, a sample of qualifications from VET, general, non-formal and higher education have been submitted for in-depth analysis. Qualifications were analysed on their content in terms of how learning outcomes corresponded to NQF level descriptors, but also on their quality, by answering a series of questions on: relevance of the qualification for the labour market; standards on which the qualification is based; structure of the qualification; involvement of labour market actors in the different stages of the qualification process; and institutional setting of the qualification. The conclusion from the inventory and analysis of qualifications were essential for the transparency of the EQF referencing and justified the decisions regarding conditional (temporary) allocation of a number of VET qualifications dependent on their future revision. ETF conducted similar inventory exercises in Serbia, Albania, and Bosnia and Herzegovina. These surveys will also inform the EQF referencing processes of these partner countries.

Since the Social Modernization Act of 17 January 2002, qualifications that become part of the repertoire national des certifications professionnelles, the French NQF register, need to be relevant, developed with the participation of social partners, and obtainable through validation of non-formal and informal learning in addition to training programmes. The national commission for professional qualifications (CNCP), checks whether all qualifications that are requesting registration in the register (sur demande) fulfil these requirements. Qualifications that are developed by the state and universities are registered automatically (de droit), but they have to fulfil these requirements as well when they are being developed.

The standards behind qualifications

As with the other parts of the qualifications quality chain, the standards behind the qualifications and how they are developed are crucial factors for quality and trust. Issues such as who is involved in their development, their relevance for the labour market (or for the purpose for which they are intended to be used), and whether they are based on learning outcomes all need to be considered. Moreover, the process of developing standards needs to be transparent.

We often hear the term ‘standard’ used in other contexts, for example ‘education standard’, ‘occupational standard’ or ‘assessment standard’, which can create confusion. To try to avoid that confusion let’s start with a very simple meaning of standard: Something that has been agreed upon by an authority or by general consent as a basis of comparison, and that is generally accepted as a basis for such comparison. From a quality assurance perspective, and for the purposes of this toolkit, it is not necessary to further define what a standard is. Instead, we need to examine how they are developed.

As mentioned above there are some criteria for standards behind a qualification that need to be met, the two most relevant being that it includes the involvement of labour market stakeholders, and is outcome-based. The main quality factor is not the how but the who. Qualifications that are developed without the systematic involvement of relevant stakeholders will, in the long run, not command trust. Learners, parents, and employers will, if they can, chose other qualifications. From the quality perspective and for quality assurance, transparent and systematic involvement by stakeholders must be part of the process to develop standards.

Outcome-based standards move the focus from input factors such as length of study to what it is that a learner should be able to know and do after a period of learning. Where and how that learning takes place is irrelevant. The (learning) outcomes serve as a declaration of content that all holders of the qualification need to meet, and the standard itself should be seen as a guarantee of good quality. Relevance is another key criterion that should be met. Labour market information, research, and analysis of needs should always be the base upon which a standard is built.

In the Turkish Qualifications Framework a distinction is made between qualifications in the Turkish Education and Training System; qualifications under the National Vocational Qualifications System and qualifications awarded by Other Bodies. Of these, the qualification standards for national vocational qualifications are the most advanced and follow a highly consistent unit-based structure. They are published on the website of the Vocational Qualifications Authority, available for everyone to see. Each unit is described in learning outcomes, performance criteria and context, and contains guidance for theoretical and practical assessment. National Vocational Qualifications are developed with stakeholders from the world of work and validated by a tripartite sectoral committee that also includes representatives of providers. The example of the bricklayers in Chapter 1 demonstrates that the quality of a qualification depends on both the development process and technical aspects. Inclusion of relevant stakeholders clearly contributes to the quality of qualifications.

ETF partner countries are trying to put in place better interconnections between the standards of qualifications, learning and assessment – well illustrated by the Competence-Based Approach to VET in Morocco; and explicitly link quality assurance of qualifications with the principles of the National Qualifications Frameworks. Another case is the Montenegro Qualification Council, which applies a common format for including qualifications in the framework, and shares responsibilities with functioning sector committees.

5. Conclusions and recommendations

The ultimate output of quality assurance is a trusted qualification. That trust takes time to build, and strategies and policies should take this into consideration and give implemented quality assurance systems time to evolve in order to produce the desired results. The success of qualifications and qualification systems, in terms of enhanced trust as well as other factors, is dependent on building infrastructures that all work towards that end. Countries need to regulate, to involve stakeholders and to put in place functional institutional arrangements to be able to have sustainable qualifications that end users see as worthwhile, and quality assurance is what brings it all together. Quality assurance links the other components of a qualification system – legislation, stakeholders and institutions. Coming back to quality assurance more specifically, we believe that to build sustainable trust it is not enough to focus on quality control. Instead, efforts need to be focused on developing a culture where the aim is to continuously learn and improve – through feedback mechanisms, recurring reviews, and collaboration between all involved. To focus only on internal quality control and external oversight of provision is not enough. Make sure that the awarding of qualifications is also quality assured – including certification, assessment, and development of standards. Use the opportunity of adopting a national qualifications framework as a tool for enhancing quality and implement new and fit-for-purpose quality assurance measures. Let’s quality assure; but as it is not an ideal world, let’s be practical.

For ETF partner countries, trusted and quality qualifications are a top priority. This concerns learners, employers, workers, education and training providers, policy makers, and the society at large. Quality assurance is particularly critical at the level where qualifications are actually delivered, learners are assessed and certificates are awarded. This is why we underscore the importance of a renewed attention to the qualifications quality chain, in particular: the standards behind qualifications, the assessment and the certification.

Recommendations

• Build a culture of quality – don’t rely on quality control.
• Anticipate the future – how changing occupations will affect qualifications needs.
• Build a systemic quality assurance approach which combines measures at various levels: policy and governance; qualifications standards; provision, assessment, validation and certification; and eventually, data and knowledge creation.
• Qualifications standards need to be:
  • Underpinned by reliable evidence defining the skills needs;
  • Based on learning-outcomes;
  • Developed in partnership by state and social-economic partners;
  • Monitored and reviewed regularly;
  • Used to underpin verified programmes, curricula and contents;
• For the critical stage of assessment, validation and certification:
  • Clearly integrate certification in VET policies and
    in QA systems;
  • Foster a common understanding of certification among stakeholders and involve them in QA processes;
  • Use appropriate and reliable mechanisms and standards;