Articulos

Evaluación formativa en producción hablada en entornos de aprendizaje virtual

Formative Assessment on Spoken Production in EFL Learners involve in E-Learning Environment

Avaliação formativa na produção falada em ambientes virtuais de aprendizagem

Morales-Morejón, Silvia; Álvarez-Peña, Marta; Mora-Aristega, Julio; Mora-Aristega, Angelica

Silvia Morales-Morejón

smorejon@utb.edu.ec

Universidad Técnica de Babahoyo Ecuador

Marta Álvarez-Peña

malvarezp@utb.edu.ec

Universidad Técnica de Babahoyo Ecuador

Julio Mora-Aristega

jmora@utb.edu.ec

Universidad Técnica de Babahoyo Ecuador

Angelica Mora-Aristega

jmora@utb.edu.ec

Universidad Técnica de Babahoyo Ecuador

Revista de Investigación en Ciencias de la Educación HORIZONTES

Centro de Estudios Transdisciplinarios, Bolivia

ISSN-e: 2616-7964

Periodicidad: Trimestral

vol. 6, núm. 22, 2022

editor@revistahorizontes.org

Recepción: 23 Noviembre 2021

Aprobación: 15 Diciembre 2021

Publicación: 03 Marzo 2022

URL: http://portal.amelica.org/ameli/journal/466/46633624663362016/

DOI: https://doi.org/10.33996/revistahorizontes.v6i22.328

Al enviar los artículos para su evaluación, los autores aceptan que transfieren los derechos de publicación a la Revista de Investigación Ciencias de la Educación Horizontes, para su publicación en cualquier medio. Con el fin de aumentar su visibilidad, los documentos se envían a bases de datos y sistemas de indización, así mismo pueden ser consultados en la página web de la Revista: https://revistahorizontes.org. Por último, la Revista se acoge en todo lo que concierne a los derechos de autor, al reglamento de propiedad intelectual del Centro de Estudios Transdiscplinarios Bolivia, el cual se encuentra en la siguiente dirección: https://www.cetbolivia.org.

License Creative Commons
Esta obra está bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional.

Modelo de publicación sin fines de lucro para conservar la naturaleza académica y abierta de la comunicación científica

ePub generado a partir de XML JATS4R

Resumen: Este estudio tiene como objetivo identificar las prácticas comunes a la hora de proporcionar retroalimentación formativa sobre la producción hablada en entornos de e-learning, al tiempo que contrasta la percepción de los profesores y alumnos. Se realizó una investigación cualitativa donde se entrevistó virtualmente a 58 alumnos y 30 docentes para identificar sus opiniones y prácticas comunes sobre el tema. Los hallazgos indican que tanto los alumnos como los profesores coinciden respecto a la importancia de la retroalimentación formativa sobre las actividades de expresión oral, sin embargo, mientras que los profesores se quejan del gran tamaño de los grupos y el tiempo limitado, los alumnos critican la calidad de la retroalimentación que han estado recibiendo, las principales objeciones están relacionadas con la falta de personalización y el exceso de generalidad en la retroalimentación.

Palabras clave: Aprendizaje virtual, EFL alumnos, evaluación formativa, producción oral.

Abstract: This study aims to identify standard practices to provide formative feedback on spoken production in an e-learning environment while contrasting the perception of teachers and students. Qualitative research was conducted with 58 learners and 30 teachers. They were interviewed virtually to identify their perceptions regarding the topic. The findings indicate that both learners and teachers have similar opinions about the importance of formative feedback on speaking activities. However, while teachers complain about the large groups and limited time, learners criticize the quality of feedback; the main objection is the lack of personalization and details on feedback.

Keywords: EFL learner, E-learning, Formative assessment, Spoken production.

Resumo: Este estudo tem como objetivo identificar práticas comuns no fornecimento do retorno formativo na produção falada em ambientes de e-learning, contrastando a percepção de professores e alunos. Foi realizada uma investigação qualitativa onde 58 alunos e 30 professores foram entrevistados virtualmente para identificar suas opiniões e práticas comuns no assunto. Os resultados indicam que alunos e professores concordam com a importância do feedback formativo nas atividades de expressão oral, no entanto, enquanto os professores reclamam do grande tamanho das turmas e do tempo limitado, os alunos criticam a qualidade do feedback que vêm recebendo, o principal as objeções estão relacionadas à falta de personalização e ao excesso de generalidade no feedback.

Palavras-chave: aluno do EFL, aprendizagem virtual, avaliação formativa, produção oral.

INTRODUCTION

The sceeries in English language learning around the world have suffered significant changes due to the COVID-19. Educators have to deal with new ways to perform their educational activities, which have imposed new challenges in their roles, and one of the most challenging activities in e-learning is to assess the achievement of the learning outcomes on spoken production appropriately. In other words, virtual education presents challenges to EFL teachers at the moment of providing assessment for learning on spoken activities. Considering that assessment for learning provides feedback on knowledge, skills, and attitudes to improve future students’ performances and learning outcomes, this process permits educators to improve students’ learning (Baehr, 2010). Hence, it is imperative to identify those aspects affecting the effectiveness of the formative assessment on spoken production in virtual setting (Boston, 2002).

Even though the usage of technology in virtual education environment has brought a variety of opportunities for the learning process, for instance, Ras, Whitelock and Kalz (2016) stated that technology provides two ways of promoting learning among students, the former implies it can be a facilitator of self-assessment and an instrument for giving feedback. The ladder refers to new opportunities viewed as strategies, resources, and tools to assess students. Other studies conducted by Nurfiqah and Yusuf (2021) demonstrated that even when teachers agree on the importance of formative assessment, virtual classes impose enormous challenges for teachers when providing effective feedback. Also, they demonstrated that there is a disconnection between specific strategies and the learning results.

The strategies and instruments applied to gather information related to learners’ spoken production on online classes are essential to determine aspects that teachers need to facilitate more support. Besides, this information help educators to make decisions about the teaching process related to keeping their planning or incorporating changes according to learners’ needs (Anderson, & Palm, 2017). Also, the assessment process requires multiple resources for getting trustworthy information from different contexts, and educators must expose learners to different kinds of situations to get real information about the development of their knowledge in the English language. The use of a single instrument cannot give accurate information about students’ performance. Hence, this study intends to identify the perspective of teachers and learners related to the effectiveness of feedback on spoken production during online classes throughout a series of interviews in which they describe their experience, including information about the strategies, procedure, and instrument. Therefore, this study aims both to identify how formative assessment enhances the speaking skills in an e-learning setting and to contrast perceptions of teachers and learners regarding formative assessment in spoken production tasks.

METHODOLOGY

This study follows a qualitative research procedure. According to Bryman (2008), “qualitative research is a research strategy that usually emphasizes words rather than quantification in the collection and analysis of data” (p. 380). Also, this study selects the participants throughout a purposively sampling procedure. The participants are students and teachers. The student participants belong to Language Center at Universidad Técnica de Babahoyo (UTB). Their ages are between 17-40 years old, they are on level 4, with a pre-intermediate domain of English, and 58 EFL learners participated in this study. Also, 38 teachers were interviewed, all of them work as EFL teachers at public universities in Ecuador, and most of them belong to the language center at UTB.

Regarding the qualitative research methodology, the instrument to gather the raw data intends to provide valuable information throughout the opinion of students and educators. Therefore, this study uses a semi-structured interview. It contains open and closed questions. The close questions are in the form of yes/no or multiple-choice, and it helps to introduce a topic, and immediately after the closed questions, the open one is presented to the interviewees. The interviews with students were done before teachers’ interviews, and it was planned on purpose to get information to direct some questions during the educator’s interviews.

The semi-structured interviews were applied virtually throughout meet sessions, and students’ participants were divided into ten groups, nine groups with 6 participants and a group with 4 participants. The semi-structured interview rested on their experiences and perceptions related to their speaking skills and the type and quality of feedback they received regarding this skill in e-learning. Moreover, teachers’ participants were divided into ten groups, nine groups with four teachers and a group with two teachers. They were divided based on their schedule availability for the virtual meeting. Additionally, their interviews were based on their experiences dealing with speaking activities and the feedback they provided to their students. Finally, the interview highlights the challenges teachers have found out when giving feedback on spoken activities.

The collected data was analyzed with the concept of grounded theory. The raw data was reviewed and analyzed to find out common aspects throughout the coding process. After that, the coding data became into categories, and it was contrasted with theoretical information to find out the connection between them.

RESULTS

Teachers´ responses

The teachers’ responses on Table 1 indicates that learners´ English level, interest, and previous knowledge seem to determine the kind of spoken activity; however, their needs, aptitude, and age are taken into consideration slightly. Additionally, teachers claimed that other aspects such as the function of the language, learning style, and self- confidence are also considered; finally, they agree that the spoken activity needs to be aligned with the lesson grammatical content.

Table 1

Aspects that teachers consider when selecting speaking activities.

Aspects | %
Language domain | 76.3%
Learners´ interests | 73.7%
Learner`s previous knowledge | 68.4%
Learner`s needs | 34.2%
Student`s aptitude | 28.9%
Age | 18.4%
Others | 5.3%

Regarding the categories considered by teachers to assess learners’ spoken productions presented on Table 2, fluency, vocabulary range, and pronunciation are preferred. Also they consider accuracy, propositional content, intonation, and syntactic density, but they are less common among teachers. Few teachers stated that communicative skills, global communicative achievement, and naturalness must be included in the analysis of spoken production.

Table 2

Aspects consider when assessing learners` spoken production.

Categories | %
Fluency | 81.6%
Vocabulary range | 76.3%
Pronunciation | 76.35
Accuracy | 50%
Propositional content | 34.2%
Intonation | 26.3%
Syntactic density | 18.4%
Others | 7.9%

According to teachers´ interviews about frequency of feedback observed on Table 3, 55% of them provide feedback after learner`s spoken productions, usually during their oral intervention or immediately after their presentations; however, another group of teachers, 36% of them claimed that they only provide feedback when curriculum required, in other words, after formal assessment or limited to 4 times during the course term. Finally, 9% of them indicate that they provide feedback when concluding a lesson, usually in a general form.

Table 3

Frequency of feedback on speaking activities.

How Often | % | Details
After learner`s participation | 55% | Every time they speak in class At the end of their presentations After any speaking exercise
Only when curriculum required it | 36% | Four times during the term Only after formal assessment
Most of the time in the form of lesson feedback | 9% | Sharing thoughts with whole class regarding their progress and needs

Concerning the type of feedback, Table 4 demonstrated that most of the teachers (64%) used general feedback given to the whole class, in which they highlight learners’ mistakes softly and drill correct pronunciation of words and phrases to allow learners to practice the correct patterns. Also, 21% of teacher participants provide personalized feedback in the form of comments, in which most of the time they use rubrics, and finally, 15% of them stated that they provided direct feedback during and after the learner’s oral production.

Table 4.

Types of feedback on speaking activities.

Type of feedback | % | How
Whole class -general feedback | 64% | Giving general feedback Explaining errors softly Making learners listening and correct patterns
Written feedback | 21% | Giving comments or sticky notes using rubrics during asynchronous
Direct feedback | 15% | During and after presentations

The Table 5 indicates that when facilitating feedback on spoken production, 45% of teachers seem helpful in sending videos and audio, in which they provide more specific and personalized feedback. Teachers used online tools such as flipgrid, padlet, video recordings with zoom, and others. At the same time, 23% of teachers also preferred to provide direct feedback; hence they use breakout rooms and synchronous sessions to accomplish this purpose. Another group of teachers mentioned that they only use online rubric creators to assess spoken production and give feedback in a more general way; finally, 18% of teachers do not use any e-tool to give feedback.

Table 5

E-tools to provide formative feedback on spoken production.

Type | % | E-tools
Audio and video | 64% | Flipgrid, Padlet, Zoom, Vocaroo, Voky, Youtube, Powtoon, Nearpod
Synchronous feedback | 21% | Breakout rooms (meet-zoom), Grading Speaking during synchronous session
Assessment resources | 15% | Rubistar, rubric maker
None | 18% | No app nor e-tools

Teachers’ voices related to positive aspects of feedback on spoken production

According to the teachers’ opinion, the benefits of feedback on spoken production can be observed from two sides. The former is related to students; it improves learners’ motivation, self-confidence in speaking a foreign language, and awareness of their progress. The ladder refers to teachers; formative feedback allows teachers to monitor students’ progress and visualize aspects that need extra attention. Some of the teachers’ opinions are presented on Table 6.

Table 6

Teachers` opinion related to positive effects of formative feedback on speaking activities.

“Formative assessment helps students to improve their speaking skill during the process. They are able to practice, correct mistakes and therefore get the goals” (teacher 10)

“It helps them improve their speaking with correct feedback, tips to improve fluency and pronunciation, and active practice” (teacher 14).

“Usually highlight the mistakes in pronunciation, intonation, use vocabulary” (teacher, 16)

“It helps them to receive a great feedback in order to correct the mistakes, improve vocabulary and fluency to have consistent dialogues.” (teacher, 20)

“Because they practice and develop their critical thinking” (teacher, 27)

Teachers` experiences related to the most challenging aspects in providing formative feedback on virtual learning are presented on Table 7. Concerning the most complicated aspects of providing feedback, teachers agree that both limited time and courses with excessive learners make it impossible to provide effective and personalized formative feedback. Besides, most students do not give enough importance to the feedback they receive. Moreover, some connectivity problems and technological issues tend to affect learners’ attention, and some web pages such as online games tend to become distracters.

Table 7

Teachers` most challenging aspects at giving formative feedback on speaking activities.

Challenges | %
Limited time | 36%
Students’ lack of interest on feedback | 21%
Students distracted /unmotivated | 14%
Connectivity/technical issues | 11%
Excessive number of students | 11%
Providing feedback in a comfortable way for learners | 7%

Learners´ responses

Regarding learners’ answers, Table 8 highlights that 52% stated that they always received feedback; also, 36% of them almost always obtain feedback, while 9% of them sometimes get it from the teacher.

Table 8

Frequency of feedback on oral production.

Never | 0%
Few times | 3%
Sometimes | 9%
Almost always | 36%
Always | 52%

Students’ opinions related to the benefits of feedback are presented on Table 9, 100% of them consider that the feedback on spoken production has helped improve their speaking skills. In addition, they claimed that when giving feedback, teachers use a range of comprehensible vocabulary, especially when explaining mistakes and providing suggestions to improve their performance. Also, concerning the most remarkable recommendation provided by teachers on the feedback, they can be observed in Table 9. Learners stated that they find more valuable videos with explicit explanations about pronunciation, speaking practice during synchronous sessions, and reinforcement throughout videos, audio, and music.

Table 9

Students` opinions about the most useful strategy to improve speaking skill.

Videos with explanations related to pronunciation | 18%
Teacher give more attention to speaking during lessons | 18%
Listening to audio, videos, and music in English | 18%
Checking recording of the classes | 13%
Web pages with explanation about pronunciation and intonation | 13%
Online dictionaries with words pronunciation | 10%
Teacher`s positive comments and motivation | 7%
Selecting difficult words and practice them | 3%

Based in results presented on Table 10, it can be stated that 100% of learners think that feedback on speaking activities has improved their spoken performance. According to their interview, 25% of them consider that they have improved their pronunciation, 22 % of students stated that they had increased their vocabulary; another important aspect they mentioned is that feedback raises learners’ confidence in their speaking ability.

Table 10

Aspects improved throughout formative feedback.

Improvements | %
Pronunciation | 24%
Increasing of vocabulary range | 22%
Improvement of self-confidence on speaking skills | 17%
Pauses during speech | 13%
Fluency | 13%
Intonation | 11%
Nothing | 0%

Learners also mention some aspects that teachers need to apply or increase while providing formative feedback despite the benefits. Learners commonly pointed out that they expect teachers promote more speaking activities; present a positive attitude and provide friendly comments while giving feedback to learners, share web pages, videos and audios in which learners can receive an explicit explanation regarding pronunciation, have more time for speaking activities and feedback and receive more extensive, explicit and personalized feedback.

Discussion

About the importance of providing feedback on students’ oral production, teachers` conception rests in the benefits for learner’s acquisition of the foreign language, since feedback raises their consciousness related to their weaknesses, learners can be aware of their needs, and also teachers obtain valuable information related to areas in which need to focus their strategies (Duque-Aguilar, 2021; Hasan; Islam, y Shuchi, 2021). Being the general outcome of the feedback in guiding and facilitating the process of EFL acquisition on learners, similar findings were obtained by Ounis (2017) studies in which he demonstrated that teachers use oral assessment feedback to facilitate and support students’ learning and their current and future verbal performances.

Besides, learners agree on teacher’s believes related to positive aspects of feedback, stating their importance on the acceptance of feedback and attitude towards error correction; this positive attitude among learners is also observed in studies carried out by Merry and Orsmond (2008) and Kavaliauskienė and Anusienė (2012), they found out that students demonstrated interest and a higher level of acceptance of feedback. Moreover, their studies found that learners tended to review feedback more than once, indicating that they considered feedback being significant for their learning.

In addition, students agreed that feedback had influenced their learning positively. They improved on aspects related to pronunciation, vocabulary, and confidence when communicating their ideas orally. Also, teachers’ strategy of including recommendations with error corrections seems to positively affect learners’ pronunciation since the audio and video resources advised by teachers to reinforce certain aspects of pronunciation are well-received by learners. Similar studies have demonstrated the positive effect of feedback throughout digital resources; Merry, et al. (2008) proved that learners perceived audio file feedback more meaningful than written feedback; also, Köroğlu (2021) demonstrated that digital formative assessment further fluency and accuracy on learners. Regarding the feedback frequency, when done after students` participation is considered a potential opportunity for improvement. According to Kavaliauskienė and Anusienė (2012), learners demonstrated a higher preference for immediate error correction, mainly because it can be assumed as connected with individual needs. However, this study showed that teachers tended to rest mainly on the feedback in a general form; in other words, even it is given individually on certain words pronunciation principally, a more structured general feedback is conducted to the whole class by pointing out common errors of the group of learners.

Complaints from teachers and learners coincide with the lack of adequate feedback due to time restrictions. Teachers claimed that time is limited when giving online classes. In addition, they consider that learners are not providing enough attention to feedback, and the existence of distracters factors limit the positive effect of feedback on oral production. On the other hand, learners maintained that they are interested in more extensive and personalized feedback, also they agree that feedback needs to include positive comments and more specific recommendations in the form of links with further information to support the error correction. All teachers agree that virtual learning environments have positive effects on education, but some challenges, too. In this point, similar results were identified by Nurfiqah et al. (2021) wherein this virtual scenario, not all students are receiving appropriate feedback and applying the adequate corrections, besides learners feels in need of “Giving more time for speaking activities and feedback on these types of activities” (student 23).

CONCLUSIONS

Nobody can deny the importance of formative feedback on learning; also, it has significant effects on learning a foreign language such as English. Concerning this study’s first objective, which is how it enhances speaking skills, it is possible to observe that formative assessment of spoken production positively affects EFL learners. Firstly, they feel motivated since the goal of learning a language is to be able to communicate ideas. Also, they perceive that it helps them improve their pronunciation and the range of vocabulary they can use to share their ideas. Finally, according to learners, these aspects positively affect the improvement in their confidence when communicating their ideas orally.

However, since education moved on from face-to-face environments to a virtual setting, teachers and students have faced challenges. Hence the second objective of this research is to contrast teachers’ and learners’ opinions regarding formative assessment on spoken production. It is observed

that one of the complicated aspects is related to the difficulty of dealing English classes with a large number of learners in a limited time; also from both learners and educators perspectives, the primary skill which is being affected is speaking since they are conscious that they do not have enough opportunity to practice this skills, and the feedback is insufficient and too general.

Despite the situation, nobody can deny that since virtual learning environment is new for Ecuadorian teachers, it will take time to adapt methodologies and identify new ways to support learners’ acquisition with more appropriate tools for providing more speaking practice and valuable feedback in the ways that learners would like to receive. Therefore, it is advisable to carry out profound studies with different online tools to identify tools to support speaking skills and provide an effective way of giving feedback on learners’ spoken production. Fortunately, both teachers and students demonstrate a positive attitude regarding virtual learning and feedback, which are observed in their comments regarding the effectiveness of well-constructed feedback.

REFERENCES

Anderson, C., & Palm, T. (2017). Characteristics of improved formative assessment practice. Education aInquiry, 8(2), 104-122. https://doi.or g/10.1080/20004508.2016.1275185

Baehr, M. (2010). Distinctions between Assessment and Evaluation. Retrieved from: http://www. pcrest2.com/institute_resources/PAI/4_1_2.pdf

Boston, C. (2002). The concept of formative assessment. Practical Assessment, Research, and Evaluation, 8(1), 9. https://doi.org/10.7275/ kmcq-dj31

Bryman, A. (2008). Social Research methods. (4 ed.). Oxford University Press. URL: https://www. academia.edu/38228560/Alan_Bryman_Social_ Research_Methods_4th_Edition_Oxford_ University_Press_2012_pdf

Duque-Aguilar, J. F. (2021). Teachers’ Assessment Approaches Regarding EFL Students’ Speaking Skill. Profile Issues in TeachersProfessional Development, 23(1), 161-177.

Hasan, M., Islam, A. B. M. S., & Shuchi, I. J. (2021). Using Mobile Based Formative Assessment in ESL/EFL Speaking. JOLLT Journal of Languages and Language Teaching, 9(1), 117-125. https:// doi.org/10.33394/jollt.v%vi%i.3449

Kavaliauskienė, G., & Anusienė, L. (2012). Case study: Learner attitudes towards the correction of mistakes. Social Technologies, 2(1), 88-101. Retrieved from: https://ojs.mruni.eu/ojs/social- technologies/article/view/147/140

Köroğlu, Z. Ç. (2021). Using Digital Formative Assessment to Evaluate EFL Learners’ English Speaking Skills. GIST–Education and Learning Research Journal, (22), 103-123. https://doi. org/10.26817/16925777.1001

Merry, S., & Orsmond, P. (2008). Students’ attitudes to and usage of academic feedback provided via audio files. Bioscience Education, 11(1), 1-11. https://doi.org/10.3108/beej.11.3

Nurfiqah, S., & Yusuf, F. N. (2021). Teacher Practice on Online Formative Assessment. In Thirteenth Conference on Applied Linguistics (CONAPLIN 2020) Atlantis Press (pp. 534-538). https://doi.org/10.2991/assehr.k.210427.081

Ounis, A. (2017). The assessment of speaking skills at the tertiary level. International Journal of English Linguistics, 7(4), p95. http://doi. org/10.5539/ijel.v7n4p95

Ras, E., Whitelock, D., & Kalz, M. (2016). The promise and potential of e-assessment for learning. In P. Reimann, S. Bull, M. D. Kickmeier- Rust, R. Vatrapu, & B. Wasson (Eds.), Measuring and visualising learning in the information-rich classroom (pp. 21–40). London: Routledge