The problems with listening during English test was only “the easy bit” of this year’s testing, Ukrainian language proficiency and ignoring of studying mathematics by prospective university students are much more worse.
The external independent evaluation-2018 was remarkable by the scandal around the audio recordings used during English listening, the unattractive results on Ukrainian and mathematics, and the fun case in Kharkiv, when the high school graduate decided not to take tests personally and found for each subject she selected a doppelganger. The first girl coped with her task, but the second one – during the testing on the history of Ukraine got busted and put paid to the scam.
In the Ukrainian Center for Educational Quality Assessment, after publishing the results of the EIT-2018, they are convinced that, despite some hardships, it can be considered successful. However, the public’s view well reflected in social media somehow doesn’t coincide with the position of officials: parents believe, that their children were the part of a regular experiment. The UCEQA denies that, saying, if it comes to listening during English test, then the children knew in advance how it would be: everything was exactly the same during a mock exam, but that time nobody had any questions.
So, as noted in UCEQA, this year the prospective university students passed one subject less than last year, that is, without the Russian language, a total of 11 subjects. The number of entrants was 323 thousand people – 100 thousand more than in 2017. Though, this year there were less “A” level students: those who received 200 points on at least one subject, turned out to be 222 people, while last year they were 303. There are only 12 of those, who have two maximum results: four in Kiev and in Kharkiv region, two more – in Lviv region, and only one student in Poltava and Volyn region.
It is clear that the Ukrainian language and literature had to be passed by all the participants of the EIT – this is the only subject that is mandatory for everyone. However, the dynamics deteriorated: if last year it was passed by 92% of test participants, then now it is only 86%. Moreover, what is especially disappointing, the Ukrainian-speaking regions showed worse results in the Ukrainian language testing than those who studied in Russian-speaking regions. As Vadym Karandii, the head of UCEQA, explained during the final press conference, this happened at the expense of graduates of vocational schools and colleges, who this year also participated in the testing, and if only pupils of schools were taken, then we would get the same 92 %.
One more compulsory test was the choice of the entrant: mathematics or the history of Ukraine. So the most popular, as in the past year, was history, as they say in the Ukrainian center: it was chosen by 168 thousand people. Mathematics was chosen by 112 thousand entrants, the English language – 89 thousand, biology – 82 thousand participants of testing. But French and Spanish languages were not very attractive for this year’s entrants: French test was passed by 566 people and Spanish by only 140.
Talking about the results, the best of them, according to UCEQA, are on biology – 92% of the participants overcame the cut score. Not bad this year’s graduates also performed in the history of Ukraine and chemistry – these subjects were overcome by 90% of those who compiled them. But mathematics and physics have been learned worse: 82% and 84% respectively passed the cut score.
For the first time, tests were passed in Braille script lettering
Vadym Karandii noted: “This year, during the EIT, there were three main innovations. They are the holding of a state final attestation in the form of EIT for students of colleges and universities, which made it possible to understand how our youth generally know the Ukrainian language, as well as listening to English and Braille testing for visually challenged people. With the request to create special conditions for them, 405 people applied, and in general, tests written in Braille were created and tested for the first time. Nine out of these entrants came to the EIT, and they showed a fairly decent result – more than 170 points, so this is a real chance for these young people.”
As for the first position, graduates of such establishments showed not very good knowledge of Ukrainian: 54% of graduates of vocational schools passed Ukrainian only at the initial level, colleges – a bit better, but worse than school graduates. It is clear that the best results here have been shown by students of high schools and specialized schools that have the right to take a competitive selection of schoolchildren. As for colleges, 10% of their graduates have only an initial level of Ukrainian, more than half have shown an average level, so here, as for me, the question is both for the selection of the students themselves and for organizing their education. Vadym Karandii also explained the scandal with English listening: they say, everything was the way it was intended, the thing is graduates, whose level of English proficiency is different, differently perceived audio recordings, and when the entrants got their results (and they were quite high), all claims have disappeared.
By the way, at the local level, they started to say that everything “was the way it was intended”, immediately after the scandal. For example, the head of the Odesa Regional Center for Educational Quality Assessment, Anatolii Anisimov, explained, that the record for the check of listening comprehension of English language was chosen “so that the knowledge was shown by students with an advanced level.” And the rest survived a real shock, which seems, nobody thought of. It is also should be noted, that many complaints were written by applicants “forgetting” that it should be done only immediately after testing.
Yurii Skoryk explains the situation from the perspective of parents in the “Facebook” network: “I want to express my opinion about the accusations, that in reality, only a few students have used the right to write an appeal in the class where the testing was conducted. The system was designed the way to minimize the negative effects of manipulations (experiments). The procedure takes into account both the psychology of stress and age peculiarities. But this time the system… outthought itself. If the state was interested in the maximum unbiased testing, then at the preparatory stage, training on the procedure and form of appeals would be conducted. So, it is irrational to make the same mistakes twice. The following year graduates of educational institutions with caring managers will come ready for force majeure. And what will the UCEQA do then, when the appeals will come in tens of thousands?”
Listening to the information about the innovations, I remembered the words of the ex-head of the UCEQA Ihor Likarchuk, said in August 2017: “Three innovations of the EIT-2018 have been made public today. Among them is the passing of EIT by graduates of colleges and vocational schools and listening during testing in a foreign language. If someone closely follows the events of the EIT, then he should remember that we were ready to implement these “innovations” in 2015. Even the appropriate decision of the MES board has been made. But then, the MES leaders were afraid of paid college rallies and expected failures in foreign language testing. They dropped this idea aside. So for EIT it is not an innovation. This is a return to what they wanted to do three years ago. And it is too far from the real innovations in today’s EIT. Because they should be, first and foremost, in the technology of evaluation and in its content. The system cannot successfully exist in a way that was ten years ago.
With regard to the third innovation – tests for university entrants with disabilities – this is a real thing. But the number of such entrants is negligible so that speaking about the systemic impact of this innovation on the EIT as an integral system is inappropriate. In general, UCEQA should make a very serious effort to counteract the devaluation of the EIT in the introductory campaign. For if, in the future, those who received 190 points will not get the scholarship, then who needs EIT? ”
According to the current UCEQA head, during the Ukrainian language testing gender disparity appeared: girls showed much better results than boys. “If we talk about a regional cut, one should keep in mind that each region has its own socio-cultural peculiarities, which also determines the level of proficiency in the Ukrainian language” explains the official. The undisputed leaders, not only in the Ukrainian but also in other subjects, are Kyiv and Lviv region. But Ukrainian-speaking Rivne region and the Ternopil region, as it turned out, lose to traditional Russian-speaking Donetsk. The reason says Vadym Karandii, is that the Russian-speaking population is mostly urbanized, and in cities, the education itself is better, and the tutors don’t live too far.
Double-level tests again?
Listening to information about bad results in the Ukrainian language testing, I remembered that for many years, the previous guidance of UCEQA (we don’t talk about Ms. Zaitseva, which was no more than a casual person in that position), strived for the introduction of two-level tests in the Ukrainian language and literature: as it is said, the deepen level for future lyric poets and more simple – for future physicists. At one time, Lilia Hrynevych, who was the first head of UCEQA, told me about her intention to introduce such an approach to the EIT in Ukrainian. Her successor Ihor Likarchuk also had a long way to this decision. And in 2015, double-level tests, not only on the Ukrainian language but also on mathematics (on the contrary, for the lyricist poets was suggested to make a simpler version), was introduced. However, they subsequently refused them, which can hardly be considered the right step.
Looking at this year’s results in the Ukrainian language and literature testing, I could not but ask the current head of the Ukrainian Center for Educational Quality Assessment, whether it is planned to return to the practice of double-level tests. Vadym Karandii explained: “The double-level testing should be, first of all, in the context of holding state final certification. The version that was introduced earlier was focused on entering the university. But higher institutions enlist students by rating, so splitting it into two different scales has absolutely no sense. As for the ADP, we talk about different state training programs. For example, in profile education programs there is a material that is not available in Standard programs. So far, I can only say about intentions, but something similar we have introduced this year in English testing. This test is suitable to rank all on a scale of 100/200, but we did not assess the ADP mark (this is an academic level, that is, regular school), not just for all, but for part of the tasks – just those that meet the requirements of the enhanced program. Those who studied on the requirements of the enhanced program in specialized classes were evaluated by a full scale, that is based, on a complete set of tests”.
But if this practice has already been introduced with foreign languages, then other subjects should not be expected to change till the next year – at best, the first attempt will be in 2020, when UCEQA will change to new EIT programs. However, the need for double-level tests at the level of the EIT is not clear, whereas for entering a higher education institution there is no question: if the future philologist needs to know perfectly the language and even the more literature, then mathematicians need to be able to write correctly, without fanaticism knowing the general trends in literature.
Yes, according to the head of UCEQA, the Ukrainian language test we have today is a bit unviable because it’s very theoretical and requires more knowledge of specific rules, writing standards, etc., and to a lesser extent, the level of language proficiency. “So, explains Vadym Karandii, the doubling of the Ukrainian language test will be in the distribution to the Ukrainian native and Ukrainian state, but at the moment, this concept is only debated and it is too early to say something specific. This will only be possible if it becomes clear that this will be laid down in the educational standard – the native or state Ukrainian language. So let’s those, who create educational standards define first.”
At first glance, such intentions look a bit strange. Why don’t we simply go back to tests of varying complexity, and recognize that rejection was false? So, we plan to return to what was before, but that we should not be accused of plagiarism, we still do it somehow, but we’ll change it. By the way, Ihor Likarchuk, the ex-head of UCEQA, has an independent opinion on the results of this year’s external independent evaluation. He wrote after the press conference, in which the results of the testing were published, on his page in “Facebook”: “Several times this morning I heard that, according to UCEQA, 50% of entrants during the EIA on mathematics didn’t pass the task of the 8th-grade program for division with fractions. The authors of the announcements almost make an ironic tragedy about it on the quality of school education, not forgetting instead to emphasize that the EIT-2018 was successful. I will not talk in details about the successful EIT now. As that is a particular topic, the fact, that it was named successful by its organizers should raise questions… But for now, I’m talking about whether it is possible to use examples of failure in the task of fractional division, speaking of the quality of school education. It is almost the same as if instead of dividing with the fractions we give an example that 70% of entrants don’t remember the formula of sulfuric acid, the height of Hoverla or the date of the last Princess Olga’s campaign. Absolutely obvious, that a graduate of a school cannot know everything from the overloaded school program. Moreover, no one needs this, that hundreds of thousands of times was proved by real-life practice.
And if so, then the question arises: “And what are the tasks in the tests? The knowledge that won’t be needed? So why is this EIA is an evaluation at all? Why UCEQA cannot refuse to test knowledge and pass to the assessment of the skills to use this knowledge to evaluate competencies?” I raise the question and I give an answer: because such tasks for EIA need to be prepared with much more attention and responsibility than those that require a simple reproduction of what been learned. By the way, in 2014, in the EIT, the use of knowledge was 20%, in 2015 it had been already 25%. And this was the line of my team, which we carried out. But then another team returned to what was before: to orient the tasks for the reproduction of knowledge. Because it’s easier for them. And it is very difficult to forget the USSR tradition. So once again I repeat: it is bad when the entrants have insufficient knowledge. But much worse is the fact, that the tools measured by this knowledge are imperfect, not to say that they are bad. It’s the same that the presence of alcohol vapor in the breath of the driver is measured by an uncertified alcohol detection device or nose of the auto inspector”.
We can’t forget about the manpower and furniture …
I advised for the experiment to put in the same class all UCEQA staff and give them “a test with the tasks for the division of fractions, knowledge of the height of Hoverla, the date of the last campaign of Princess Olga. You can add several tasks to wise cases of use of punctuation marks in the Ukrainian language. I’m not talking about listening in English.” It would be interesting to see the results of this test.
In his comment to DUMKA Fedir Kramarenko, the former teacher of physics, he works as a methodologist on subjects, EIT and its monitoring in Zhashkiv district of the Kiev region: “Why on earth we had that English listening this year! I have been responsible for many years for the preparation and conduct of testing in the Zhashkiv district. It is even hard to imagine from one side to another what a complex of work it is. To ensure that I can arrange EIT in a district, I have to organize testing places that need to be provided by a certain number of competent professionals. Where to get well-trained staff to ensure that the EIT is at a high organizational level? Nowadays no one can give the answer to this question – bluff your way”.
As for the listening, the former teacher of physics explains, if we impose the same requirements on anyone – rural, urban, this year, last year’s graduate, then everyone should have the same conditions. Fedir Kramarenko says: “It’s about some typical devices, with one mark, of one quality and not relying on the fact that the headmaster will find some player there and will scroll through those disks. And then if I have an ear for music, then I hear very well. And as someone has some problems with hearing, it’s worse. One says: everything sounds good here, and the other says: nobody will hear that. Our children, for example, went to Uman district to pass English, and then there were complaints that it was not clear enough and the quality of the equipment reproducing sound during the audition was bad”.
According to the educator, it is necessary to provide a technical basis, so that the entrants could not have an opportunity to try to outwit: “And when they won’t be able to outwit? When this player is equipped with some indicators that prove that there is enough sound output, to take some kind of super Japanese player is not enough – this equipment should be special, which testifies that it works properly.”
However, according to the responsible for the EIT in the Zhashkiv district, such issues don’t exhaust the problems with the EIT. He says: “We are forced to use the rooms where the primary school classes are located in certain cases, it is clear that there are no desks there for the 11th grade. That is, by ourselves, we cannot always provide the entrants – participants of the EIT with the proper furniture in connection with the fact, that in recent years, no one takes care of these issues. So, besides the questions, related to the tasks of the tests, there is still a staff and technical problem. At first glance, they are not visible, but if we don’t pay attention to them, they won’t go anywhere…”
And one does not need to be a seer to say, that such a situation isv only in the Kyiv region. Most likely, the farther from Kyiv, the more problems similar to these. At the same time, EIT is almost the only true educational reform, which allows those entrants, who otherwise would not have expected it, to get a student card. The main conclusion of 2018 is evident: the system for conducting EIT needs modernization. And only the time will tell whether the current leaders will take into account all the acute angles of this complex mechanism.
Text by Larysa Vyshynska
