Introduction

This study was part of a research project investigating the sleeping habits of young people who play games online. No properly validated Portuguese instrument designed for use in a European context existed, so we translated the Pittsburgh Sleep Quality Index (PSQI). The PSQI is often used in comparative research. In order to overcome the lack of properly validated instruments to evaluate sleep in Portugal, we assessed the factor structure of a Portuguese version of the PSQI (P-PSQI). Our aim was to determine whether the factorial structure of the original PSQI was replicated in young adolescents from a different culture from that in which the original instrument was validated.

The PSQI measures subjective sleep quality and sleep disturbances during the previous month. Poor sleep is associated with various health problems, reduced human performance, mood swings and low academic achievement (Gaultney, 2010). Both psycho-socio-cultural and biological factors can contribute to poor sleep quality. A study conducted in Japan by Kaneita et al. (2007) concluded that young people who slept between 7 and 9 hours a day had the best health. Sleeping less than 7 hours or more than 9 hours seems to have a negative effect on mental health.

Morrison, McGee e Staton (1992) reported that poor sleep was associated with higher levels of anxiety, depression, attention deficit and behavioural problems, regardless of gender. The American Psychological Association (APA, Division 54) and the Society for Developmental and Behavioral Pediatrics have collaborated with various journals to address the dearth of scientific literature on sleep and associated behavioural and health problems. The objective of the collaboration was to promote research on issues associated with sleep quality in children and adolescents with developmental disorders. In recent years, thanks to this partnership, we have witnessed a small increase in research on issues related to sleep quality. If lack of scientific research on sleep quality is a problem at international level, then in the problem is even more acute in Portuguese-speaking countries. The few studies that are carried out tend to use instruments that have not been validated for use in Portuguese-speaking populations.

This is why we chose to study the PSQI. The PSQI, which is the most commonly used sleep research questionnaire, consists of 19 items, grouped into seven components; responses are given on a 0 to 3 scale. Total scores range from 0 to 21 points. Scores above 5 suggest some kind of impairment of sleep quality (Almeida, 2013; Buysse et al., 1989). Component 1 measures overall subjective quality of sleep. Component 2 deals with sleep latency, the time it takes for an individual to fall asleep. Component 3 measures the duration of sleep, i.e. the number of hours of sleep per night, and component 4 evaluates the usual sleep efficiency. Component 5 (daily dysfunction due to sleepiness) relates to diurnal somnolence and disorders that may manifest during the day. Component 6 identifies whether the subject takes sleeping medication or not and Component 7 measures dysfunctional sleep/ sleep disorders and is intended to identify possible somnolence felt throughout the day (Almeida, 2013; Bertolazi, 2008; Buysse et al., 1989).

According to Bertolazi (2008), Buysse et al. (1989) and Cunha (2006), the PSQI is an instrument for assessing sleep quality that is widely used in various age groups and cultural contexts. It has been translated and validated in Taiwan (Tzeng, Fu, & Lin, 2012), Israel (Shochat, Tzischinsky, Oksenberg, & Peled, 2007), Italy (Curcio et al., 2013), Pakistan (Hashmi et al., 2014), Persian (Farrahi, Nakhaee, Sheibani, Garrusi, & Amirkafi, 2009; Moghaddam, Nakhaee, Sheibani, Garrusi, & Amirkafi, 2012), Korea (Sohn, Kim, Lee, & Cho, 2012), Greece (Bertolazi et al., 2011; Kotronoulas, Papadopoulou, Papapetrou, & Patiraki, 2011). It has also been used with college students in Nigeria (Aloba, Adewuya, Ola, & Mapayi, 2007), in Brazil (Bertolazi, 2008; Bertolazi et al., 2011; Neto et al., 2014), in an inter-racial study with women over 70 years old (Beaudreau et al., 2012), with Indian students (Manzar et al., 2015) and with a group of pregnant women from Peru (Zhong, Gelaye, Sánchez, & Williams, 2015). No Portuguese (European) version was identified in the literature review.), in an inter-racial study with women over 70 years old (Beaudreau et al., 2012), with Indian students (Manzar et al., 2015) and with a group of pregnant women from Peru (Zhong, Gelaye, Sánchez, & Williams, 2015). No Portuguese (European) version was identified in the literature review.

Although there are numerous translations and validations, to date the PSQI has not been validated in a Portuguese population. Researchers working with Portuguese samples tend to use the translation developed for use in the Brazilian population (Bertolazi, 2008). Two Portuguese groups have announced that they were carrying out validations of the PSQI (Marques et al., 2013; Gonçalves, Peralta, Ferreira, & Guilleminault, 2015), but we have not found any such validation published in a scientific journal or deposited in any of the publically accessible Portuguese bibliographic repositories. Given the urgent need for a validated European Portuguese instrument for evaluating sleep quality, and in view of the characteristics of the PSQI that make it very useful for research and clinical application, we decided to validate a Portuguese version in a Portuguese sample, using confirmatory factor analysis.

According to the conceptualisation of the basic theoretical perspective for the evaluation of sleep quality, the purpose of our study was to verify that the European Portuguese version of the PSQI (P-PSQI) shared the factor structure of the original when used in young people.

Methods

Our sample was drawn from the school population in the interior northern region of Portugal, using probabilistic sampling criteria randomisation by clusters to ensure the heterogeneity of the subjects. The sample consisted of 252 young people (113 girls; 139 boys) who participated voluntarily and anonymously. The subjects’ ages ranged from 12 to 19 years, with a mean of 15.3 years (SD = 1.93).

Instruments

The questionnaire used to collect data was divided in two parts: first, personal data and second, sleep habits during the last month (assessed using the P-PSQI). The first part provided data on the independent variables for the study. The second captured information about students’ habitual pre-bedtime behaviour and any sleep problems.

The P-PSQI consists of 19 items. It consists of seven components whose scores vary on a scale of 0 to 3. Total scores vary between 0 and 21 and values over 5 indicate impaired sleep quality.

Statistical Analysis

We performed the confirmatory factor analysis (CFA), using the maximum likelihood estimation method. This technique is based on structural equation modelling (SEM) and is used to analyse the relationships between the set of variables that make up the structural model. CFA is a commonly used statistical procedure for which there are several statistical packages e.g. LISREL, AMOS, EQS, SAS, CALLIS, Mplus and R. CFA enables one to determine whether a proposed model is a good fit to the data submitted for analysis (León, 2011).

We used the SPSS (Statistical Package for Social Sciences-23) with the AMOS extension. First we calculated descriptive statistics (mean and standard deviation) for the sample to obtain the measures of central tendency.

A set of indices of fit were used to assess with the proposed model was a good fit to the data: chi-square (χ2), χ2/df ratio and root mean square error of approximation (RMSEA) and its 90% confidence interval (90% CI) were calculated. Lower values indicate better model fit.

Although the χ2/df ratio is used as an index of fit there is no consensus about a criterion for adequate fit, with values below 3 being suggested (Aroian & Norris, 2005). The CFI evaluates the suitability of the model in relation to the independent model, and values higher than 0.90 indicate adequate fit (Byrne, 2010).

The GFI measures the proportions of variance and covariance jointly explained by the model. Values above 0.90 indicate adequate fit (Byrne, 2010). The RMSEA is a measure of the discrepancy between the estimated and observed matrices; values between 0.08 and 0.1 indicating a mediocre model, and when superior to 0.1 indicates a poor suitability, consequently it is recommended the possibility of rejection of the model (Byrne, 2010; Marôco, 2014). We used the Akaike Information Criteria (AIC) to compare obtained factorial models; lower values suggest better model fit when compare to the others (Byrne, 2010).

The comparative fit index (CFI), goodness of fit index (GFI) adjusted goodness of fit index (AGFI) and AIC indicate the degree of parsimony/simplicity of the model tested (Fernandes & Vasconcelos-Raposo, 2009; León, 2011). To evaluate model fit we used the following cut-off values: χ2/df ≤ 5.0, RMSEA ≤ .08 and CFI, GFI and AGFI ≥ .90. Recently it has been argued that more stringent criteria should be used for these indices, namely: χ2/df ≤ 3.0, RMSEA ≤ 0.06 and CFI, GFI and AGFI ≥ .95 (Hu & Bentler, 1999).

Results

Table 1 shows descriptive statistics (mean and standard deviation) for all components of the scale. Preliminary analysis indicated that the variables ‘sleep medication’ and ‘sleep efficiency’ had high skewness and kurtosis, so on the basis of recommendations in the specialist literature (Marôco, 2014; Nevitt & Hancock, 2001; Yuan, Chan, & Bentler, 2000) we used the bootstrap technique.

Table 1
Mean, standard deviation, skewness and kurtosis values for the Sleep variables.

Sleep variables	Mean (M)	Standard deviation (SD)	Skewness w/bootstrap	Kurtosis w/bootstrap
Subjective quality	0.87	0.724	0.020	0.048
Latency	1.17	0.948	0.001	0.013
Duration	0.22	0.582	0.084	0.737
Usual efficiency	0.34	0.770	0.023	0.223
Disorders	0.90	0.463	0.023	0.027
Use of sleeping pills	0.21	0.675	0.115	2.009
Daytime dysfunction	0.83	0.830	0.019	0.036
Total PSQI	4.55	2.872

Taking into account that in the original version we could not find an instrument for the Portuguese population (Portugal), we performed a confirmatory factor analysis (CFA). The results obtained suggested corrections and the following were made for the following covariances (e3 ≤ - ≥ e4 and e5 ≤ - ≥ e7). After these corrections the fit indices were as follows: χ2/df = 2.085, GFI = 0.972, AGFI = 0.934, CFI = 0.911, RMSEA = 0.67 [0.029 - 0.104], AICsat = 56.0. These values show that the P-PSQI is robust and therefore suitable for research and clinical use.

Although the initial analysis of fit indicated that the PSQI was robust, closer analysis of the squared Mahalanobis distances indicated the presence of several outlying observations (p1 and p2 ≤ .001). When these observations were excluded the fit indices were as follows: χ2/df = 1.194, GFI = 0.983, AGFI = 0.959, CFI = 0.980, RMSEA = 0.029 [0.001 - 0.076], AICsat = 42.0.

Figure 1
The final model of the P-PSQI

We observed that there was a negative relationship between component 6 (sleeping medication use) and the component 1 (overall sleep quality). Removal of component 6 improved model fit significantly and the final values for the fit indices were: χ2/df = 1.014, GFI = 0.990, AGFI = 0.979, CFI = 0.990, RMSEA = 0.008 [0.000 - 0.081], AICsat = 42.0. The model’s composite reliability (CR) was 0.564.

DISCUSSion

The purpose of this study was to examine the factorial structure of the proposed dimensions of P-PSQI. To the best of our knowledge this is the first study to assess the validity of a Portuguese version of the PSQI in the Portuguese population.

The original PSQI was intended to evaluate seven dimensions of sleep: (1) overall sleep quality, (2) latency, (3) duration, (4) usual efficiency, (5) disorders/ dysfunctions, (6) use of sleeping medication and (7) daytime dysfunction. We were unable to confirm the validity of the original seven dimensions as we eliminated dimension 6 (use of sleep medication). This result should was not a surprise, partly due to the age of our sample, which associated to the fact this dimension was negatively correlated with the overall sleep quality. Both conditions justify the elimination of the item.

In the course of this study, and after determining the psychometric characteristics of this questionnaire through both an exploratory factor analysis, we confirmed the results with a confirmatory factorial analysis (AFC). This procedure allows a more appropriate analysis besides allowing validating the factorial structure and confirming that it is the same, even in different samples.

Our results showed that most of our sample experienced good sleep quality. Given the dearth of sleep research using CFA it was difficult to compare our findings with other studies. Our results led us to discard dimension 6 of the P-PSQI. This decision was not without precedent as questions had already been raised about dimension 6 (Zhong et al., 2015). Zhong et al. (2015) reported that removing the use of sleep medication component improved model fit. Based on their findings and our own results we excluded dimension 6 from our model and observed a clear improvement in overall model fit: χ2/df = 1.014, GFI = 0.990, AGFI = 0.979, CFI = 0.990, RMSEA = 0.008 [0.001 - 0.081], AICsat = 42.0. Our findings thus corroborate those of Zhong et al. (2015). The fit indices for the revised six-factor model exceeded the most stringent published criteria (Browne & Cudeck, 1993; Hu & Bentler, 1999) and indicate that a six-factor version of the P-PSQI, without the use of sleep medication component, is quite robust. The factorial structure that resulted from the elimination of component 6 (taking sleeping medication), shows that for the age group studied, the PSQI should be considered appropriate for nomothetic studies with samples made up of young people.

Comparing our results with those of Zhong et al. (2005) we found that the CFI and RMSEA values were very similar: in our study CFI = 0.911 and RMSEA = 0.067, whereas in the study by Zhong et al. (2005) CFI = 0.91 and RMSEA = 0.04.

We paid special attention to the various levels of equivalence bias. Instruments developed in one cultural (and/ or linguistic) must be translated and validated before they can be applied in another context. To overcome these requisites, we used theory-oriented validation procedures, namely CFA, as our intention was to confirm that the P-PSQI shared the factor structure of the original PSQI.

The types of adaptation domains considered were: 1) conceptual, which was subdivided into two orientations, theory and adaptation; 2) culture, subdivided into orientation for the terminological/ factual rigor and for the normative criteria; 3) language, subdivided into linguistic and pragmatic rigor. Finally, we paid particular attention to how the questionnaire was presented to respondents. We treated theory-oriented validation as the most important form of validation. The remaining aspects of validation were taken into account in the work preceding statistical testing of the validity of the proposed seven-factor model. This was accomplished through the processes that preceded the statistical analysis to test the hypothesis of validation of the factorial structure proposed initially. That is, if the theoretical requirements for the use validation of questionnaires were taken into consideration during the performance of the necessary procedures that precede the use of CFA and consequently verify if the same number of factors and items would be maintained in a sociocultural different sample. Other aspects that need to be taken into consideration were to confirm if the factorial loadings would be equivalent in the different cultural groups (German and Portuguese), and if the same items explained the same level of construct variability (Byrne, 2010). The results for the P-PSQI, as evidenced by the observed fit indices, did not confirm the validity of the seven-factor structure as we eliminated dimension 6.

Overall the CFA and the reliability results demonstrated that this Portuguese version of the PSQI has excellent psychometric properties and suggested that in adolescents a six-dimension version should be used, rather than the seven-dimension version originally proposed. The decision to discard the sleep medication component in analysis of data from young people should not be considered extraordinary, as young people tend not to use this type of medication. We therefore do not regard the elimination of this dimension as a threat to the theoretical foundation of the PSQI. To the best of our knowledge this study makes the P-PSQI the only sleep research instrument validated for use in Portuguese young people. Until other instruments are subjected to the same validation procedures, the P-PSQI should be the instrument of choice for research into sleep quality in Portuguese samples.

The main limitation of the PSQI is that it requires a somewhat complex treatment to obtain the scores for each dimension. We believe the questionnaire needs to be simplified to make it suitable for both nomothetic research and clinical use. Although we eliminated dimension six in our analysis of data from school-age children, we believe that in general this dimension should be retained, as its unreliability may be limited to the age group investigated in this study. Another limitation of the study relates to the sample, which was limited to schools in the interior north of Portugal. Our research should be replicated in other regions of the country to demonstrate the wider validity of the P-PSQI. For the future, we suggest prioritising attempts to validate the P-PSQI in other age groups. Because our sample was limited to school students we do not know whether similar results would be observed in other age groups. Future studies should use the full seven-component scale and seven-factor structural model.

conclusion

The Portuguese version of the PSQI presented here is, to the best of our knowledge, the first sleep quality scale to be validated in Portuguese adolescents. Our results indicate that the P-PSQI is a valid and robust instrument, since it maintained its theoretical factorial structure initially proposed based on the German sample. Although the theoretical robustness of the questionnaire was confirmed, we were unable to confirm the seven-factor structure of the original PSQI in our school-age sample; the sleep medication factor had to be eliminated from the structural model to optimise fit.

Despite the limitations, the P-PSQI was shown to be a good instrument to measure sleep quality, and suited for use in sleep research with Portuguese adolescents.

Referências

Almeida, G. A. S. (2013). Distúrbios no padrão do sono e fadiga em pacientes submetidos à cirurgia oncológica gastrointestinal. (MSc. dissertation). Retrieved from http://bdtd.uftm.edu.br/bitstream/tede/120/1/Dissert%20Guilherme%20AS%20 Almeida.pdf

Aloba, O. O., Adewuya, A. O., Ola, B. A., & Mapayi, B. M. (2007). Validity of the Pittsburgh sleep quality index (PSQI) among nigerian university students. Sleep Medicine, 8(3), 266-270. https://doi.org/10.1016/j.sleep.2006.08.003

Aroian, K. J., & Norris, A. E. (2005). Confirmatory factor analysis. In B. Munro (Ed.), Statistical Methods for Health Care Research (pp. 351-375). Philadelphia: Lippincott Williams & Wilkins.

Beaudreau, S. A., Spira, A. P., Stewart, A., Kezirian, E. J., Lui, L., Ensrud, K., Redline, S., Ancoli-Israel, S., & Stone, K. L. (2012). Validation of the Pittsburgh sleep quality index and the Epworth sleepiness scale in older black and white women. Sleep Medicine, 13(1), 36-42. https://doi.org/10.1016/j.sleep.2011.04.005

Bertolazi, A. N. (2008). Tradução, adaptação cultural e validação de dois instrumentos de avaliação do sono: escala de sonolência de Epworth e índice de qualidade de sono de Pittsburgh (MSc. dissertation). Retrieved from http://www.lume.ufrgs.br/bitstream/handle/10183/14041/000653543.pdf?sequence=1

Bertolazi, A. N., Fagondes, S. C., Hoff, L. S., Dartora, E. G., Miozzo I. C. S., Barba, M. E. F., & Barreto, S. (2011). Validation of the brazilian portuguese version of the Pittsburgh sleep quality index. Sleep Medicine, 12(1), 70–75. https://doi.org/10.1016/j.sleep.2010.04.020

Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230-258.

Buysse, D. J., Reynolds III, C. F., Monk, T. H., Berman, S. R., & Kupfer, D. J. (1989). The Pittsburgh sleep quality index: a new instrument for psychiatry practice and research. Psychiatry Research, 28(2), 193-213. https://doi.org/10.1016/0165-1781(89)90047-4

Byrne, B. M. (2010). Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming (2000, 2nd ed.). New York: Routledge

Cunha, M. C. B. (2006). Qualidade do sono em diabéticos do tipo 2 (MSc. dissertation). Universidade de São Paulo, São Paulo.

Curcio, G., Tempesta, D., Scarlata, S., Marzano, C., Moroni, F., Rossini, P. M., Ferrara, M., & Gennaro, L. (2013). Validity of the italian version of the Pittsburgh sleep quality index (PSQI). Neurological Sciences, 34(4), 511-519. doi: 10.1007/s10072-012-1085-y

Farrahi, J., Nakhaee, N., Sheibani, V., Garrusi, B., & Amirkafi, A. (2009). Psychometric properties of the Persian version of the Pittsburgh sleep quality index addendum for PTSD (PSQI-A). Sleep and Breathing, 13(3), 259-262. doi: 10.1007/s11325-008-0233-3

Fernandes, H. M., & Vasconcelos-Raposo, J. J. B. (2009). Análise factorial confirmatória do TEOSQp. Psicologia: Reflexão e Crítica, 23(1), 92-101.

Gaulteney, J. (2010). The prevalence of sleep disorders in college students: impact on academic performance. Journal of American College Health, 59(2), 91-97. http://dx.doi.org/10.1080/07448481.2010.483708

Gonçalves, M., Peralta, A. R., Ferreira, J. M., & Guilleminault, C. (2015). Sleepiness and motor vehicle crashes in a representative sample of portuguese drivers: the importance of epidemiological representative surveys. Traffic Injury Prevention, 16(7), 677–683. http://dx.doi.org/10.1080/15389588.2015.1013535

Hashmi, A. M., Khawaja, I. S., Butt, Z., Umair, M., Naqvi, S. H., & Jawad-Ul-Haq. (2014). The Pittsburgh sleep quality index: validation of the Urdu translation. Journal of the College of Physicians and Surgeons Pakistan, 24(2), 123-126.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. http://dx.doi.org/10.1080/10705519909540118

Kaneita, Y., Ohida, T., Osaki, Y., Tanihata, T., Minowa, M., Suzuki, K., Wada, K., Kanda, H., & Hayashi, K. (2007). Association between mental health status and sleep status among adolescents in Japan: a nationwide cross-sectional survey. Journal of Clinical Psychiatry, 68(9), 1426-1435.

Kotronoulas, G. C., Papadopoulou, C. N., Papapetrou, A., & Patiraki, E. (2011). Psychometric evaluation and feasibility of the greek Pittsburgh sleep quality index (GR-PSQI) in patients with cancer receiving chemotherapy. Supportive Care in Cancer, 19(11), 1831-1840. doi: 10.1007/s00520-010-1025-4

León, D. (2011). Análise fatorial confirmatória através dos softwares R e Mplus. (Tese de Licenciatura). Retrieved from http://www.lume.ufrgs.br/bitstream/handle/ 10183/31630/ 000784196.pdf

Manzar, M. D., Moiz, J. A., Zannat, W., Spence, D. W., Pandi-Perumal, S. R., BaHammam, A. S., & Hussain, M. E. (2015). Validity of the Pittsburgh sleep quality index in indian university students. Oman Medical Journal, 30(3), 193- 202. doi: 10.5001/omj.2015.41

Marôco, J. (2014). Análise de Equações Estruturais: Fundamentos Teóricos, Software & Aplicações (2010, 2nd ed.). Pêro Pinheiro: Report Number –Análise e Gestão de Informação, Lda.

Marques, D., Gomes, A. A., Meiavia, A., Salgueiro, A., Ribeiro, C. C., & Dischler, J. (2013). Reliability and initial validation of the Pittsburgh sleep quality index, european portuguese version: a preliminary study in a sample of higher education students. Sleep Medicine, 14(1). http://dx.doi.org/10.1016/j.sleep.2013.11.316

Moghaddam, J. F., Nakhaee, N., Sheibani, V., Garrusi, B., & Amirkafi, A. (2012). Reliability and validity of the persian version of the Pittsburgh sleep quality index (PSQI-P). Sleep and Breathing, 16(1), 79-82. doi: 10.1007/s11325-010- 0478-5

Morrison, D. N., McGee, R., & Staton, W. R. (1992). Sleep problems in adolescence. Journal of the American Academy of Child & Adolescent Psychiatry. 31(1), 94- 99. https://doi.org/10.1097/00004583-199201000-00014

Neto, J. B. B., Germain, A., Mattos, P. F., Serafim, P. M., Santos, R. C. M., Martini, L. C., Suchecki, D., & Mello, M. F. (2014). Psychometric properties of the brazilian version of the Pittsburgh sleep quality index addendum for PTSD (PSQI-A). Revista Brasileira de Psiquiatria, 36(4), 330-335. http://dx.doi.org/10.1590/1516-4446-2013-1225

Nevitt, J., & Hancock, G. R. (2001). Performance of bootstrapping approaches to model test statistics and parameter standard error estimation in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 8(3), 353-377.

Shochat, T., Tzischinsky, O., Oksenberg, A., & Peled, R. (2007). Validation of the Pittsburgh sleep quality index hebrew translation (PSQI-H) in a sleep clinic sample. The Israel Medicine Association Journal, 9, 853-856.

Sohn, S., Kim, D. H., Lee, M., & Cho, Y. (2012). The reliability and validity of the korean version of the Pittsburgh sleep quality index. Sleep and Breathing, 16(3), 803-812. doi: 10.1007/s11325-011-0579-9

Tzeng, J. I., Fu, Y., & Lin, C. (2012). Validity and reliability of the taiwanese version of the Pittsburgh Sleep Quality Index in cancer patients. International Journal of Nursing Studies, 49(1), 102-108. https://doi.org/10.1016/j.ijnurstu.2011.08.004

Yuan, K., Chan, W., & Bentler, P. M. (2000). Robust transformation with applications to structural equation modelling. British Journal of Mathematical and Statistical Psychology, 53(1), 31-50. doi: 10.1348/000711000159169

Zhong, Q., Gelaye, B., Sánchez, S. E., & Williams, M. A. (2015). Psychometric properties of the Pittsburgh sleep quality index (PSQI) in a cohort of Peruvian pregnant women. Journal of Clinical Sleep Medicine, 11(8), 869-877. doi: 10.5664/jcsm.4936

Autor notes

José Vasconcelos-Raposo. Quinta de Prados, Apt. 1013, ECHS-1, Gab. 2.27, 5000-881 Vila Real, Portugal. E-mail: jvraposo@utad.pt