COMUNICACIÓN TÉCNICA

Tolerable limits of fecal coliforms in the conservation of vacuum-packed beef stored at 0° C

Límites tolerables de coliformes fecales en la conservación de carne bovina envasada al vacío, almacenada a 0° C

Camacho-Rioja, María Deisy; Asfura-Telchi, Jorge; Huarachi-Chirilla, Cinthia

Camacho-Rioja, María Deisy

mariacamacho@uagrm.edu.bo

Gabriel René Moreno Autonomous University Bolivia

Asfura-Telchi, Jorge

Gabriel René Moreno Autonomous University Bolivia

Huarachi-Chirilla, Cinthia

National Service of Agricultural Health and Food Safety. Bolivia

Journal of the Selva Andina Animal Science

Selva Andina Research Society, Bolivia

ISSN: 2311-3766

ISSN-e: 2311-2581

Periodicity: Bianual

vol. 10, no. 2, 2023

directoreditoranimalscience@gmail.com

Received: 01 March 2023

Accepted: 01 August 2023

Published: 01 October 2023

URL: http://portal.amelica.org/ameli/journal/198/19846441984644009/

DOI: https://doi.org/10.36610/j.jsaas.2023.100200096x

Selva Andina Research Society

License Creative Commons
This work is licensed under Creative Commons Attribution-NonCommercial 4.0 International.

Non-profit publishing model to preserve the academic and open nature of scientific communication

ePub generated from XML JATS4R

Abstract: pH and temperature are fundamental parameters for the storage and maintenance of beef quality. The purpose of the present work was to evaluate the shelf life of meat from 120 samples of 10 steers, during 3 months of storage at 0° C and vacuum packed. For this purpose, a measurement was made 48 h post mortem in 3 pH ranges: <5.8, 5.9 and 6 to 7 to quantify Escherichia coli and mesophilic aerobes. Regarding the sensory factor, flavour, colour, odour and texture of the meat were evaluated. Samples were evaluated monthly. The statistical analysis corresponded to the randomized block design, applying the Tukey test at 5 % and 1 %. The mesophiles, were 6.8x10⁶ to 10x10⁶ CFU/g, E. coli that developed between 1.75 to 2.5 CFU/g. Bacterial growth showed values of 5.9 and 6 to 7 pH, unfavourable values according to Bolivian standard 310017. In the evaluation of E. coli, there was no significant statistical difference (p>0.05), however, aerobic bacteria showed a significant difference in the months of study (p<0.01). The sensory analysis did not show variations until the 2nd month of conservation, deteriorating afterwards with pH higher than 5.9 in meat stored under vacuum for 3 months, which means that its sensory and microbiological characteristics are not within the normal parameters. It is concluded that meat with intermediate and high pH should not be vacuum packed or stored for more than 3 months, as it loses its organoleptic characteristics. However, meat with low pH and stored at low temperature maintained its quality for 3 months, therefore it is considered that pH and temperature are important factors to control microbial growth and maintain meat quality.

Keywords: Faecal limits, preservation, pH, beef, Escherichia coli..

Resumen: El pH y la temperatura son parámetros fundamentales para el almacenamiento y mantenimiento de la calidad de la carne bovina. El propósito del presente trabajo fue evaluar la vida útil de la carne proveniente de 120 muestras de 10 novillos, durante 3 meses de almacenamiento a 0° C y envasada al vacío. Para ello se hizo una medición 48 h post mortem en 3 rangos de pH: <5.8, 5.9 y 6 a 7 para cuantificar Escherichia coli y aerobios mesófilos. Respecto al factor sensorial se evaluó sabor, color, olor y textura de la carne. La evaluación de las muestras se realizó mensualmente. El análisis estadístico correspondió al diseño de bloques al azar, aplicándose la prueba de Tukey al 5 y 1 %. Los mesófilos, fueron 6.8x10⁶ a 10x10⁶ UFC/g, E. coli que desarrolló entre 1.75 a 2.5 UFC/g. El crecimiento bacteriano presentó valores de 5.9 y 6 a 7 de pH, valores desfavorables según la norma boliviana 310017. En la evaluación de E. coli no presentó diferencia estadística significativa (p>0.05), sin embargo, las bacterias aeróbicas presentaron diferencia significativa los meses de estudio (p<0.01). El análisis sensorial no evidenció variaciones hasta el 2 mes de conservación, deteriorándose después con pH mayor a 5.9 en carne almacenada al vacío durante 3 meses, esto significa que sus características sensoriales y microbiológicas no están dentro los parámetros normales. Se concluye que la carne con pH intermedio y alto no debe ser envasada al vacío ni almacenada por más de 3 meses, ya que pierde sus características organolépticas. Sin embargo, la carne con pH bajo y almacenada a baja temperatura mantuvo su calidad durante 3 meses, por ello se considera que el pH y la temperatura son factores importantes para controlar el crecimiento microbiano y mantener la calidad de la carne.

Palabras clave: Límites fecales, preservación, pH, carne bovina, Escherichia coli..

Introduction

The preservation and export of meat is an essential process in the meat industry, involving the application of techniques and measures that guarantee its quality, product safety during transport and storage. Meat preservation refers to the implementation of methods and technologies that make it possible to prolong shelf life, avoiding deterioration and preserving its organoleptic characteristics.

According to Luzardo-Butria 1 , vacuum packaging helps prevent oxidation and the development of microorganisms by eliminating oxygen from the package 2 and can also maintain the sensory quality of the meat, such as color and texture, for a prolonged period of time 3 .

The use of controlled or modified atmospheres is also common in the preservation of meat, these techniques involve the modification of the gaseous composition of the environment in which the meat is found, with the objective of prolonging its shelf life, Brody et al. 4 , can help inhibit the growth of microorganisms and delay the oxidation of meat components 5 .

Refrigeration allows meat to be kept at low temperatures, which reduces the development of microorganisms and delays spoilage processes. On the other hand, freezing involves reducing the temperature of the meat to below its freezing point, which stops microbial activity and preserves the quality of the product 6 .

Meat exports imply complying with the requirements, sanitary and quality regulations established by the importing countries. According to the Ministry of Rural Development and Lands of the Plurinational State of Bolivia, there are specific protocols for exporting meat to different countries 3 .

Livestock farmers and government representatives from Bolivia and China indicated that the Processing Plants of Frigorífico del Oriente S.A (FRIDOSA) and Matadero Frigorífico Santa Cruz (FRIGOR S.A.) were the first to be registered for the export of deboned meat to the Asian country, with an export of 40000 t in 2019 and 117000 t projected for the year 2025 7 .

Annual consumption in 2019 of beef in Bolivia was 217687 t, and per capita annual consumption of 21 kg, the main producing departments were Santa Cruz and Beni, among the cities with the highest consumption, Santa Cruz, La Paz and Cochabamba with 71 % of the national total. Bolivia needs markets to generate added value to its economy by diversifying the supply of meat processing plants with specialized vacuum processed cuts. Another strategy of the meat industry in growing development is to offer safe, hygienic, good quality products 2 .

Vacuum-packed metabolically activated foods, such as meats, continue respiratory activity, consuming a small amount of oxygen present in the product's tissues, which increases the vacuum producing carbon dioxide and water vapor. The increase of CO. in the package has advantages as an inhibitor against many microorganisms 1,4 .

Quality is a very complex term, and is not the only definition valid for all levels of meat production, involving characteristics of carcass composition as determinants of market value, and the most recent ones consider its nutritional, organoleptic, technological and sanitary hygienic properties 8 .

The purpose of the present study was to determine the tolerable limits of fecal coliforms in vacuum-packed meat preserved under refrigeration, by means of total mesophilic aerobic bacteria counts, sensory analysis of the organoleptic characteristics of beef and shelf life of beef in 3 pH ranges.

Materials and methods

The research work was carried out at FRIDOSA, located 10 km east of the city of Santa Cruz. For the study, carcasses were obtained from 10 Nelore breed steers that entered the deboning room, 120 samples were collected and the pH was measured 48 h post mortem.

A quantitative experimental investigation was proposed with 2 variables under study: pH of beef in 3 ranges: pH < 5.8 (normal), 5.9 (intermediate) and 6 to 7 (high) 8 and its storage for a period of 3 months at 0±1° C 4 , vacuum-packed in cardboard boxes. The samples were taken in FRIDOSA's deboning room. The organoleptic characteristics of flavor, color, odor and texture were evaluated in cooked meat from the beginning to the end of the test. The raw material used was meat from 10 beef steers. We worked with the front cut (shoulder), which corresponds to the triceps brachii muscle, 30 samples were selected for each pH level, within 48 h after slaughter. A pH-meter with a Testo 230 electrode was used to measure the samples. Subsequently, the shoulder cut obtained with a sterilized knife was divided into 4 to make a total sample of 120 cuts 8 . For the statistical analysis, the randomized block design was applied, followed by the minimum significant difference (LSD) and Tukey tests with a significance of 5% and 1% 9,10,11 .

In the laboratory, the microbiological analysis method Petrifilm Plate NB 32020 12,13 was used for the CFU count of Escherichia coli. The total mesophilic aerobic count was carried out according to NB 32003 and NB 32016 ¹⁴ , ¹⁵ .

Sensory analysis (flavor, color, odor and texture) was evaluated on 3 cm thick samples, cooked according to the ISO 22000 1 Food Safety Management System standard (Table 1).

Table 1

Parameters for sensory analysis of meat according to pH ranges 9

Parameters | Initial | 1 month | 2 months | 3 months
Taste | Crude | Not applicable | — |  |  |
Stew | Beef and veal | Own | Own | Slightly acidic | Acid
Color | Crude | Bright reddish | Reddish | Reddish | Opaque red | Brown
Stew | Cooked meat | Own | Own | Brown | Brown
Odor | Crude | Meat | Meat | Meat | Slightly acidic | Residual
Stew | Meat | Meat | Meat | Residual | Residual
Texture | Crude | Normal | Normal | Normal | Slightly soft | Soft
Stew | Normal | Normal | Normal | Slightly soft | Soft

Results

Microbiological analysis. The total mesophilic aerobic bacteria count from day 0 to 3 months showed a significant difference (p < 0.05) between the 3 pH levels and the lowest development was at pH < 5.8 with 6.8x10⁶ CFU/g 4,8 (Figure 1). When comparing the CFU count of vacuum-packed meat with the limits set by the Bolivian IBNORCA standard 1x10⁵ CFU/g 10 for fresh, refrigerated and/or frozen red meat, it was observed that the meat samples presented counts above the permissible level, the limit being 10x10⁶ CFU/g, so the risk of spoilage is higher 11 .

Figure 1
Development of mesophilic aerobes in CFU/g according to pH range

Figure 2
Development of E. coli in CFU/g according to pH range

E. coli at pH 5.9 presented the lowest CFU count with 1.75 and the highest at pH 6 to 7 with 2.25 CFU (Figure 2), a value that is within the levels established by the NB 310017 standard. No significant statistical difference by month was observed in any of the 2 microorganisms (p>0.05).

Sensory analysis. The organoleptic characteristics of flavor, color, odor and texture were evaluated in cooked meat from the beginning to the end of the test (Table 2). At pH < 5.8 there were no variations in the first month, but in the second month a change in color was observed (it became an opaque red) and in the third month there were variations in the three parameters.

At pH 5.9 from month 2 the samples showed variations in 3 parameters and in month 3 a change in odor (Table 2). At pH 6 to 7 from month 2, there were variations in color and odor, and in month 3 in texture and odor. At pH 5.9 and pH 6 to 7 after 2 months of storage, there were evident signs of deterioration (Table 2).

The E. coli CFU count showed low values under the conditions of pH, storage, time and temperature in which the study was carried out 16 , indicating that no significant growth was observed for E. coli in the meat cuts stored at 0 and 4° C for 3 months.

Table 2

Sensory evaluation with pH: < 5.8, 5.9 and 6 to 7 in vacuum-packed meat stored at 0 ± 1° C

Parameters | pH ˂ 5-8 |
Initial | 1 month | 2 month | 3 month
Taste | Crude | Not applicable |  |  |  |
Cocido | Beef and veal | Own | Own | Characteristic matured meat | Characteristic matured meat
Color | Crude | Bright reddish | Reddish | Reddish | Opaque red | Brown
Cocido | Cooked meat | Own | Own | Opaque red | Brown
Olor | Crude | Meat | Meat | Meat | Characteristic matured meat | Residual
Cocido | Meat | Meat | Meat | Meat | Characteristic matured meat
Texture | Crude | Normal | Normal | Normal | Slightly soft | Soft
Cocido | Normal | Normal | Normal | Slightly soft | Soft

Table 2

Sensory evaluation with pH: < 5.8, 5.9 and 6 to 7 in vacuum-packed meat stored at 0 ± 1° C (Continued)

pH 5.9 | pH 6 a 7 |
1 month | 2 month | 3 month | 1 month | 2 month | 3 month
Own | Characteristic matured meat | Acid | Own | Slightly acidic | Ácid
Reddish | Opaque red | Brown | Reddish | Opaque red | Brown
Own | Brown | Brown | Own | Marrón | Brown
Meat | Characteristic matured meat | Residual | Meat |  | Residual
Meat | Residual | Residual | Meat |  | Residual
Normal | Slightly soft | Soft | Normal | Slightly acidic | Soft
Normal | Slightly soft | Soft | Normal | Residual Slightly soft | Soft

Discussion

In the study carried out, it was found that meat with intermediate pH (5.9) and high pH (6 to 7) should not be vacuum-packed or stored for a period of 3 months because it does not maintain its organoleptic characteristics that ensure quality and safety for human consumption 17 .

The storage of vacuum-packed cuts with a pH < 5.8 and kept at 0° C contributes to the quality of the product, controls microbial development and maintains its sensory characteristics in commercially acceptable conditions for 3 months, which allows its transport to distant destinations 17-20 .

FRIDOSA uses innovative techniques for the preservation of vacuum-packed meat products, thus improving the shelf life and quality of its cuts, for which it conducts studies on texture, water retention, microbial development and color 5 .

Research carried out at FRIDOSA is related to the control of meat quality with the pH of live muscle, which is normally above 7, with the pH of the meat after slaughter reaching values of 5.4 to 5.7 in normal meats.

The pH of meat has an effect on its organoleptic characteristics such as color and water retention capacity, but has only a slight effect on flavor and tenderness, which is why it is difficult to predict changes in the latter through pH 6 .

Maturation time has a significant influence on pH. Frozen meat does not show increases in pH in relation to storage time. Vacuum packaging was able to neutralize the increase in pH, but at the time of packaging, the product should not have a pH higher than 5.85. The variation of pH values is the effect of many factors, some of which are specific to the animal (genetics, metabolism, susceptibility to stress, etc.), but normally the most relevant ones have to do with the environment in which the animal was handled and its carcass during the 24 h before and after slaughter 8 .

In relation to mesophytic bacterial, significant variations in CFU/g were observed in the samples analyzed, coinciding with Martinez et al 7 . who mentioned that these variations are due to factors such as the final pH of the meat, temperature and storage time; it is worth noting that pH values <5.8 showed low microbial development.

In the sensory analysis of the 1 month cooked meat, no variations were observed in the organoleptic characteristics at the 3 pH levels, but in the 2nd and 3rd months it showed signs of deterioration unfit for human consumption at pH > 5.9 21 . At high pH, the risk of deterioration (protein degradation, putrefaction) is greater. Meat and meat products with pH above 6.0 are particularly risky 1, ²² .

A fundamental parameter for ensuring the shelf life of fresh cuts is the refrigeration temperature, which has a direct effect on the speed of microbial growth; however, strict control of microbial load and temperature is not sufficient to achieve a limited marketing period ²³ .

In this sense, cuts stored at 4° C presented physicochemical and appearance alterations in month 3 pH <5.4 16 , loss of vacuum and presence of odors, while at 0°C no alterations were perceived until month 5 .

The color change occurred from month 2 at pH levels > 5.8. Meat color is the first quality aspect that directly influences consumer preferences18 and the decision of whether or not to buy beef, the first one perceived at the time of purchase 24- ²⁶ .

Cited Literature

1. Luzardo-Butria A. Vida útil de la carne: influencia del envasado y sistema de producción [Internet]. Córdova: Sitio Argentino de Producción Animal; 2017 [citado 20 de marzo de 2023]. 22 p. Recuperado a partir de: https://www.produccion-animal.com.ar/informacion_tecnica/carne_y_subproductos/198-LUZARDO-BUIATRIA.pdf.

2. Exportaciones de carne bovina y sus preparaciones [Internet]. Instituto Boliviano de Comercio Exterior. 2022 [citado 20 de marzo de 2023]. Recuperado a partir de: https://ibce.org.bo/ibcecifras/index.php?id=1081.

3. Ministerio de desarrollo rural y tierras del estado plurinacional de Bolivia. Protocolo de exportación china [Internet]. La Paz: Ministerio de desarrollo rural y tierras del estado plurinacional de Bolivia; 2019 [citado 20 de marzo de 2023]. 5 p. Recuperado a partir de: https://fegasacruz.org/wp-content/uploads/2020/07/Protocolo_de_exportacion_china.pdf.

4. Brody AL, de Fernando Minguillón GDG, Ordoñez Perea JA, Ascensio Pérez MA. Envasado de alimentos en atmósferas controladas, modificadas y a vacío. Zaragoza: Acribia S.A; 1996.

5. Certificaciones [Internet]. Fridosa. 2023 [citado 20 de marzo de 2023]. Recuperado a partir de: https://fridosa.com.bo/es/certificaciones/.

6. Swatland HJ. Evaluación de la carne en la cadena de producción. Zaragoza: Acribia S.A; 1995.

7. Martínez E, Soto S, Chávez J. Influencia de los lípidos sobre parámetros sensoriales de la carne. Ciencia y Tecnología Alimentaria [Internet]. 2002 [citado 3 abril 2023]; 9:40-3. Recuperado a partir de: https://www.researchgate.net/publication/303826037_Influencia_de_los_Lipidos_Sobre_Parametros_Sensoriales_de_la_Carne/references.

8. Braña Varela D, Ramírez Rodríguez E, Rubio Lozano MS, Sánchez Escalante A, Torrescano Urrutia G, Arenas de Moreno ML, et al. Manual de análisis de calidad en muestras de carne [Internet]. Querétaro: Centro Nacional de Investigación Disciplinaria en Fisiología y Mejoramiento Animal; 2011 [citado 26 de marzo de 2023]. Folleto Tecnico No. Recuperado a partir de: https://www.academia.edu/22857285/_Manual_de_An%C3%A1lisis_de_Calidad_en_Muestras_de_Carne.

9. Fisher RA. Statistical methods for research workers. In: Kotz S, Johnson NL, editors. Breakthroughs in Statistics. New York: Springer Series in Statistics; 1992. p. 66-70. DOI: https://doi.org/10.1007/978-1-4612-4380-9_6.

10. Tukey JW. Comparing individual means in the analysis of variance. Biometrics 1949;5(2):99-114. DOI: https://doi.org/10.2307/3001913.

11. Kramer CY. Extension of multiple range tests to group means with unequal numbers of replications. Biometrics 1956;12(3):301-7. DOI: https://doi.org/10.2307/3001469.

12. Bryant M. Microchemical methods. In: Latimer GW, Jr, editor. Standard method performance requirements (SMPRs(r)). Oxford: University of Oxford; 2023. DOI: https://doi.org/10.1093/9780197610145.003.502.

13. AOAC International. Standard Method Performance Requirements (SMPRs(r)) for Nontargeted Testing (NTT) of Ingredients for Food Authenticity/Fraud Evaluation of Honey [Internet]. Rockville: AOAC International; 2020 [cited Mar 22, 2023]. 3 p. Retrieved from: https://www.aoac.org/wp-content/uploads/2020/06/AOAC_Quant_Micro_Methods_SMPRv4_FINAL.pdf.

14. 3M PetrifilmTM. Product instructions [Internet]. Minneapolis: 3M Company; 2022 [cited Mar 22, 2023]. 108 p. Retrieved from: https://multimedia.3m.com/mws/media/1015782O/petrifilm-rapid-aerobic-ifu.pdf.

15. Aerobic plate count [Internet]. U.S. Food and Drug Administration. 2001 [cited Mar 22, 2023]. Retrieved from: https://www.fda.gov/food/laboratory-methods-food/bam-chapter-3-aerobic-plate-count.

16. García AL, Brugnini G, Rodriguez S, Mir A, Carriquiry J, Rufo C, et al. Vida útil de carne fresca de res envasada al vacío a 0 C y+ 4 C. PAyDS 2015;4:27-45.

17. Golden MC, Wanless BJ, David JRD, Lineback DS, Talley RJ, Kottapalli B, et al. Effect of equilibrated ph and indigenous spoilage microorganisms on the inhibition of proteolytic Clostridium botulinum toxin production in experimental meals under temperature abuse. J Food Prot 2017;80(8):1252-8. DOI: https://doi.org/10.4315/0362-028X.JFP-17-012.

18. Mancini RA, Hunt MC. Current research in meat color. Meat Sci 2005;71(1):100-21. DOI: https://doi.org/10.1016/j.meatsci.2005.03.003.

19. Yang X, Zhang Y, Zhu L, Han M, Gao S, Luo X. Effect of packaging atmospheres on storage quality characteristics of heavily marbled beef longissimus steaks. Meat Sci 2016;117:50-6. DOI: https://doi.org/10.1016/j.meatsci.2016.02.030.

20. Duran A, Kahve HI. The effect of chitosan coating and vacuum packaging on the microbiological and chemical properties of beef. Meat Sci 2020;162:107961. DOI: https://doi.org/10.1016/j.meatsci.2019.107961.

21. Schmidt Hebbel H. Carne y productos cárnicos: su tecnología y análisis [Internet]. Santiago de Chile: Fundación Chile; 1984 [citado 21 de marzo de 2023]. 11 p. Recuperado a partir de: https://repositorio.uchile.cl/bitstream/handle/2250/121407/schmidth05.pdf.

22. Egan AF. Lactic acid bacteria of meat and meat products. Antonie Van Leeuwenhoek 1983;49(3): 27-336. DOI: https://doi.org/10.1007/BF00399507.

23. Tirado J, Paredes D, Velazquez G, Torres JA. Microbial growth in refrigerated meat products. Cienc Tecnol Aliment 2005;5(1):66-76.

24. Joo ST, Kim GD, Hwang YH, Ryu YC. Control of fresh meat quality through manipulation of muscle fiber characteristics. Meat Sci 2013;95(4):828-36. DOI: https://doi.org/10.1016/j.meatsci.2013.04.044.

25. Godoy Gaitán RM. Análisis químico, evaluación sensorial y valor proteico de una galleta de harina de trigo (Triticum aestivum) y harina de arveja dulce (Pisum sativum) [tesis maestría]. [Guatemala]: Universidad de San Carlos de Guatemala; 2010 [citado 26 de marzo de 2023]. Recuperado a partir de: https://biblioteca-farmacia.usac.edu.gt/tesis/MANA4.pdf.

26. Saláková A. Instrumental measurement of texture and color of meat and meat products [Internet]. Brno: Department of Meat Hygiene and Technology; 2012 [cited May 2, 2023]. 34 p. Retrieved from: http://www.maso-international.cz/wp-content/uploads/2013/04/maso-international-2012-2-page-107-114.pdf.

Notes

Source of financing: This study was financed by the Faculty of Veterinary Sciences of the Universidad Autónoma Gabriel Rene Moreno through an agreement with Frigorífico del Oriente S.A. (FRIDOSA).

_________

Conflicts of interest: The authors declare that this research was conducted at the Universidad Autónoma Gabriel René Moreno de Santa Cruz (Andrés Ibáñez Province) and presents no conflicts of interest.

_________

Acknowledgments: The authors gratefully acknowledge the financial support of FRIDOSA and the Universidad Autónoma Gabriel Rene Moreno. They also thank the students involved in this research.

_________

Ethical considerations: Approval of the research by the Directorate of Veterinary Research and the Research Committee of the Veterinary Medicine Department of the Universidad Autonoma Gabriel René Moreno (UAGRM) followed the guidelines established by these bodies.

__________

Authors' contribution to the article: María Deisy Camacho Rioja, experiment planning, statistical analysis, systematization and interpretation of information. Jorge Asfura Telchi, systematization and interpretation of the information, revision of the document. Cinthia Huarachi Chirilla, systematization and interpretation of the information, revision of the document.

________

Research limitations: The authors point out that there were no limitations in the present research work.

_________

Article ID: 130/JSAAS/2023.

________

Editor's Note: Journal of the Selva Andina Animal Science (JSAAS). All statements expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, editors and reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author notes

* Contact address: Gabriel René Moreno Autonomous University. Faculty of Veterinary Sciences. Veterinary Public Health. University City, Av. 26 de febrero. Tel: +591-33542014, fax: +591- 3537676. Santa Cruz, Plurinational State of Bolivia.

Maria Deisy Camacho-Rioja E-mail address: mariacamacho@uagrm.edu.bo

Alternative link

http://www.scielo.org.bo/scielo.php?script=sci_arttext&pid=S2311-25812023000200096&lng=es&nrm=iso&tlng=en (html)