Vadym Karandii on peculiarities of the EIE-2019

The EIE (External Independent Evaluation) results have already become a definite indicator of the quality of education and give a great reason to wonder why only every tenth entrant chooses physics or chemistry, why the school systematically “pushes out” boys and why the future is in innovative education. The new Deputy Minister of Education and Science Vadym Karandii, the former director of the Ukrainian Center for Educational Quality Assessment told how a thorough analysis of test results will help improve approaches in the secondary education system.

Why does the school “push out” the boys?

Mr. Karandii, you’re supervising secondary education in the relevant department. Is it possible to improve the quality of the EIE by analyzing the results?

What the whole public sees when watching the EIE is only part of the information. It needs to be analyzed deeper. For example, now we are giving not only summary indicators of students’ educational achievements but also an indicator of the fulfillment of a specific test task by students of a class and school compared to how they coped with it in the city, district, region, Ukraine as a whole. This information is available to teachers, school principals on our website and allows analyzing exactly where the gaps in knowledge are.

Independent assessment is being improved, the number of open type tasks increases. In general, the power of the EIE is much greater than the influence of regulatory documents, inspections. Perhaps that is why this exam has become mandatory and shows the current situation in the quality of education. Thanks to the EIE, everything is exposed: we can look at the results of each educational institution, draw conclusions and correct the situation.

From year to year, the lowest results of Ukrainian language among students are in the Transcarpathian, Ivano-Frankivsk, Rivne, Chernivtsi regions. What problems does the EIE exposes in these regions?

The test of the Ukrainian language is specific – this is the test on native language and not a language proficiency test. The results of graduates of these areas are determined by a number of aspects: social, demographic, national, and others. In Transcarpathia, more ethnic minorities live, for whom Ukrainian is not native, the majority of students are not in Ukrainian-speaking surroundings at all.

At the same time, the results differ depending on the type of educational institutions. This year is a striking gender gap: the guys showed three times worse test results of the Ukrainian language. The gain came from the vocational school, where boys mostly study. This means that after the ninth grade, the boys are forced out of the system of general secondary education and go to vocational schools.

What is the reason?

Mainly in physiological, age, psychological features. Since the behavior of boys is often destructive, the teacher may not respond to such students. As a result, the student is pushed out of the learning process, living an independent educational life.

The problem was in the shadow for a long time, nobody noticed it. Vocational schools were created earlier to solve the problems of the secondary school – they sent the worst students that the teachers could not or did not want to cope with, to the vocational school. Therefore, the system should be rebuilt, and analyzed why this happens, at what stage, how to react to it. It is necessary to understand that we are preparing a worker, not for a year or even 10 years. Therefore, it is necessary to change the approaches in the system of secondary education – it must keep up to date.

Last year, for the first time, the State Final Examination was introduced in the form of the EIE of Ukrainian language for graduates of vocational schools and colleges. But in some of them, Ukrainian is not taught at all in the first year of study!

Unfortunately, there are not only vocational schools, but also colleges, where half of the graduates have not even overcome the “passed/not passed” threshold. And this is a problem. Vocational school leaders mostly do not analyze it, they think the children will receive blue-collar jobs. In 2019, graduates of vocational schools and colleges will pass the Ukrainian and choose between mathematics or the history of Ukraine, and in 2020 – between these and other specific subjects.

In general, it is clear from the test results that the children did not receive elementary knowledge in fifth-seventh grades at school! It was revealed when the journalists found information in the summer that only half of the graduates, who were passing mathematics, coped with the elementary mathematical equation 2+2:2. But this is the task for seventh grade!

When will the New Ukrainian School “germinate”?

Teachers of the fourth grades are “sharpened” at the formal SFE (State Final Examination). How do you see the change in this evaluation of knowledge?

At the time we initiated this work, the center was supported by the Ministry of Education and Science and the Renaissance Foundation. After all, the SFE should be conducted to monitor educational activities. Previously, there was no means of collecting and analyzing information about the educational achievements of students. The SFE system should do this, but it has been reduced to a formality.

Mass SFE with such a tool that can create the Ukrainian Center for Educational Quality Assessment, apparently, will not exist. Only the format will change. Most likely, this will be a measure of responsibility for each school. But we will periodically carry out control evaluation in the form of monitoring the study at the state and regional levels.

The tests are waiting for the ninth graders

Should the students of the ninth grade prepare for the EIE?

We are thinking with colleagues about the fact that it is impossible to simply transfer the approaches of the EIE, which the graduates of eleventh grades pass, for ninth-grade students. This should be a completely different test, where there are more classic tasks that require a detailed answer. We need to build a system that will have positive feedback on the organization of training. Otherwise, the teacher will not be guided by methodological recommendations, curriculum, but into the means by which the assessments are carried out at the end. Now we are still working on a formula for combining the subjects and their quantities.

The new law “On Education” provides for the introduction of a specialized high school. Is it real to develop a subject matter test?

Now there is a discussion in the circle of experts. On the one hand, examinations for the ninth-graders should become a specific filter for determining which subject a ninth-grade graduate should study. On the other hand, at this age, it is still difficult for a young person to decide on a future profession. And here it is important not to rush, because there will be a situation like with vocational schools: we will sift children out with this test.

But there are many cases when a child studies in a remote rural school, where there are no appropriate conditions for learning. And by changing the environment you can get a different result. Now we are talking about the career guidance test and finding your place just for nin^th-graders. This is still a project.

Education on the “demarcation zone”

The demand for secondary education in Ukraine is increasing remotely among students from uncontrolled territories of the east.

According to the EIE results, the achievements coming from the Donetsk and Luhansk regions are not problematic. Graduates from the territories outside the government control (TOGC) compile independent testing. Interestingly, Russian-speaking applicants have a better test result in the Ukrainian language than Ukrainian-speaking ones. These are those, who are behind the “demarcation zone”, they are more motivated because they really want to enter the Ukrainian university. Also, the Russian-speaking population that prevails in the south and east of Ukraine is more urbanized. Urban education institutions are potentially stronger than rural ones.

Were there any special registration conditions for entrants from the TOGC in 2018 and what should children prepare for this year?

We give young people from the occupied territories of the east an opportunity to preserve their anonymity: do not indicate the territory of origin. Some make a note to keep correspondence for them at the regional center for the evaluation of the quality of education. At the same time, they keep track of all the information through their electronic account. Moreover, now electronic rooms work in the electronic app “My EIE”.

Every year we try to contact the relevant services and inform potential participants of the EIE, who will cross the demarcation line, that conditions are created for them to accelerate the passage of all procedures.

In the expert environment, there is a suggestion to create for children, young people from the TOGC preparatory departments, where they would prepare for the taking of the EIE. Is this necessary?

I think yes. They need to go through a certain adaptation. Children from the TOGC really need an adaptation period. It’s good if students have our teachers in that territory – Ukrainians by the soul.

Is it known how many young people from temporarily occupied territories have become students of Ukrainian universities, colleges, vocational schools?

Under the conditions of military conflict, not every applicant from the east wants to show the territory of their origin. Now they can be traced only by obtaining a certificate, knowing which schools provide distance learning and issue certificates to such students. This is actually one thread. Occasionally we use it.

In Crimea, especially in the early years of the conflict, people, who traveled from the occupied peninsula were subjected to various meanness. We even had a war of calendars: they appointed their unified state examination on our EIE. Their graduates, not wanting to lose the Russian certificate, first compiled the Unified State Examination, and then they were ready to go to the mainland, to take the EIE. Therefore, we conducted for them even the third testing session (the second additional one). Then we decided to postpone the tests as much as possible. As far as we have feedback coming from the east, this is fine: for one entry they can pass the EIE and submit documents.

Why haven’t we heard about a foreign language?

In 2018, many applicants passed a foreign language, because, with the introduction of listening, the results of last year could not be submitted.

A foreign language is the only one of all subjects that applicants are increasingly choosing, says a growth in the number of high school graduates. The History of Ukraine is growing, mathematics, unfortunately, is falling. There was a slight increase from 2017-2018 when more universities required mathematics. Applicants see that their achievements in mathematics are insignificant, and they go to pass the history. Young people do not want to take the EIE in subjects, which require great efforts to learn. Only every tenth school graduate now chooses physics or chemistry.

Here is a complex problem. Physics and chemistry are not supported for decades. Cabinets are being updated, and these two subjects without experiment – laboratory, demonstration – are impossible. A significant role is played by the state of the economy and the labour market. Since many professions that are based on physics or chemistry, are not in demand in the market.

The indignation of society was caused not by the difficulty of listening, but by the quality of its reproduction. Why did this happen?

The listening unit was quite easy, it was elementary. Here another problem manifested itself. Most students were not really ready to listen to the information presented in this way. The second problem is the creation of conditions for such an exam. The Ukrainian centre is not the owner of the premises, where they are tested. The room must have acoustics. And not only for English listening unit. At school, acoustics should always be excellent for a high-quality educational process where all communication is built on voice.

We clearly stated in the recommendations that during the period of the audio recording, the windows must be closed to avoid extraneous sounds. Imagine: a testing point, 15 audiences, where a recording with a difference of several seconds sounds simultaneously. With bad acoustics and soundproof, the participants heard a sound not only in their audience (classroom) but also in the next room. Some wrote: I had a feeling of echo. These problems were revealed exactly during the EIE, they need to be tackled.

What will you do in 2019, how to technically ensure the conditions for a foreign language test?

We will take into account as much as possible. First, in terms of sound recording: we’ll see, the recording at which frequencies is less subjected to acoustic distortion. Those schools, where there were most complaints, we will have to give up and go to others. Also, acoustic testing should be done not in one audience, as it was this year, but in several at once.

Now the ratings of educational institutions according to the results of the EIE are popular. How much do they reflect the quality of the study?

I would not use them. Any ratings that are based on the EIE are incorrect. After all, they do not show the whole picture. Here’s how to compare a city school with a rural one with a rather complicated social environment, where there is a sufficiently large number of unemployed or those, who went to work abroad? This environment greatly influences the child, since many responsibilities are assigned to it.

But in any regional city, there is a lyceum, where conditions for competitive selection are created. Therefore, it is impossible to compare these places. We have information on each institution, the one who wants to find a good school for their child, can use the section “Regional data”.

150 points for doctors and not only them

Until now, the debate continues about cutoff 150 points for future physicians. Is it effective to regulate enrollment at a particular speciality?

This threshold doesn’t really show anything, since the rating scale is applied, so it’s impossible to write out the knowledge requirements for exactly 150 points. This will be justified only if the criterion scale is established. What happened as a result? Applicants, who did not choose biology, preferred geography.

Medical universities have achieved their goal: they did not pick the best of the best, but simply did not allow the worst. In medicine, it more or less worked, although it created a negative effect. Because through the introduction of restrictions, general secondary education is deformed. If the applicants do not choose biology, physics, chemistry, mathematics, they will prefer other subjects – humanitarian. What to do then?

The Ministry of Justice wants to set a passing score of 150 for future lawyers. But this should not be done. This will result in the deformation of the school, the professional environment. There is no such selection anywhere in the world, there’s one limitation – the presence or absence of a high school diploma.

Will tests be always rated or will they become criterion someday?

If used for university entrance, they are usually based on a rating scale. If this is a school test, it must be a criterion. Even the mark that is currently set for the EIE is double: for those, who enter universities, this is a scale of 100-200 points, for those, who graduate from school, the scale from 1 to 12 points is the result of the SFE. For ninth-graders, only the initial scale will be used, when the assessment is linked to the level of academic achievements which the graduate has. Here, the rating scale is inappropriate, since there is no competition.

There is a version to introduce in universities a minimum competitive cutoff 135 points. But this is actually the sum of various subjects: Ukrainian, History, and foreign language. And the applicant cannot calculate in advance on which subject he will score these points and on which will not. This is more like a manual control situation: we want to manually control this system, but it should be self-regulating. Here, for the establishment of 150 points, the EIE system will be self-regulating – applicants, who do not select certain subjects will choose others. And in order for it to be self-regulating, it is necessary to raise the requirements for general secondary education. School certificate should not be a common good. It turns out that the certificate will not be issued only when the student has not attended more than 50% of the study time.

That is, no reference to the knowledge?

No. School certificates leveled. A graduate of the school, who comes to compile the SFE receives 1 point only for coming to the SFE – there is no other assessment in the scale. This is not a joke, but a reality. But the question is, whether it can be recognized as a complete secondary education in the sense that it should be?

In September 2007, when Ukraine only built the EIE system, I went to Paris. We were taken to one of the schools, which was the examination point, similar to our schools, which at the time of EIE become test points. There were a lot of young people there. We asked why. It turned out that in France only 70% of graduates take exams for their bachelor’s degrees from the first time. And the bachelor’s degree is a complete secondary education. And then they enter universities. Therefore, their admission to the university is a certificate. And so in many European countries.

Test for higher education

Are all future bachelors interested in introducing the EIE to get a diploma of this educational level?

It is assumed that graduates at a certain qualification level will have to pass the Unified State Qualifying Examinations. This is a kind of analogue of the EIE, but it will be conducted by the university itself for a certain profession. Such exams will replace state exams that recognize the qualifications. The center is not involved.

We work in part of such exams as a single professional test for admission to the Master of Laws, which was introduced in 2017. It is also the only entrance exam of foreign languages for students, who want to join the magistracy. In 2018, it was first introduced for those, who entered the humanities, social, behavioural sciences. Every year the list of areas and specialities increases.

In 2019, the list is more than doubled: public, state administration, management. Higher education at the master’s level has to acquire a person, who can receive information in a foreign language from primary sources. The foreign exam is simple ‒ under the EIE programs, in fact, the equivalent of an exam for school graduates, even a simplified one ‒ without a written part.

What the applicants should expect in 2019?

Changes will be minimal. There are several reasons. The first, this is the last year, when we work on the existing programs of external independent assessment. From 2020, the programs will be different, different, but not drastically ‒ they should go to two-level tests: in 2020 ‒ for the Ukrainian language, 2021 ‒ for mathematics.

In addition, there are other global tasks in Ukrainian Center for Educational Quality Assessment, which we are also working on. Since the EIE, in the way proven to be effective, certain efforts should be directed to other areas: the SFE for nin^th grade, monitoring – both international PISA studies and internal monitoring studies, in particular, the quality of primary education. Another challenge for the center is to participate in the process of certification of teachers.

The EIE for teachers

Do teachers also need to prepare for certain tests to undergo voluntary certification? Will it be conducted by the centre or postgraduate education institutions?

The certification is voluntary, it is built on motivation. The teacher, who successfully passes it, receives an appropriate surcharge, and the second ‒ has the right to participate in certain types of work. Certification consists of three parts: self-analysis, independent assessment (two-part test) and expert assessment of the teacher’s practical activities. On testing ‒ it is a question of the Ukrainian center, the creation of the format of this testing, but the approach itself will be close to the EIE. The big problem now is the expert assessment of the teacher ‒ this is a complicated procedure. As for me, it should be clearly formalized. Postgraduate education institutions can be good performers ‒ they need to be provided with a normal methodology, assessment technology, there should be scales, a clear understanding of the criteria.

Today, certification is planned for only one group ‒ primary school teachers. Since this can be done only when there is a clear idea of the activities of the teacher. Now a clear standard is formulated only for primary school teachers, the rest are in development. Voluntary certification should be introduced in October this year.

Interview by Inna Kovaliv