Year 2: Implementation
As in the first year, the medical Spanish section was offered over the first five Fridays of the fall semester and was divided into 1-hour lectures followed by 2-hour group sessions in the afternoon. However, the traditional lectures were digitized and placed onto a Learning Management System (LMS) for the students to access at their own pace. The lessons remained the same as in the previous year.
All students were required to self-assess their Spanish-speaking ability. As before, this was done on a scale of 1 (fluent/native speakers) to 5 (no Spanish knowledge at all). In addition to this, we also administered a course pre-test to students on the LMS. The course pre-test included 20 questions of varying difficulty. Self-assessment and pre-test scores were significantly correlated (Spearman's rho, p<.01), suggesting that student self-assessments were relatively accurate (data not shown). We used both the self-assessment scores and the scores on the course pre-test to divide the students into groups for the afternoon sessions. We divided the groups in the same way as in the previous year.
All students were required to take a pre-lesson assessment for each of the five lessons. The lesson assessment questions were presented in a random order and were written at a basic-to-intermediate level. Students who scored 100% on the pre-lesson assessment were excused from watching the digital lecture. This was implemented to respect the time of the advanced students and reward them for serving as peer tutors. Students who did not obtain a perfect score were required to watch the digital lecture and then complete a post-lesson assessment with a passing score of 70% to receive credit for attendance. After the 5 weeks, students completed the course post-test to measure their progress.
The last change we made from Year 1 was to divide students into groups of four to work together in interactions with Spanish-speaking simulated clients. During these scenarios, the students introduced themselves to their clients in Spanish, took a history, performed a physical exam, and communicated diagnostic and treatment information to their clients. These interactions took place after the conclusion of the 5 weeks of medical Spanish, and each group had two simulated client interactions on a single day. As in the previous year, a student's grade for the medical Spanish section of Clinical Correlates was based on attendance.
Year 2: Student Outcomes
The course pre-test was designed to separate students into five distinct skill levels. The pre-test consisted of 20 questions: four very basic questions, six low-intermediate questions, six high-intermediate questions, and four advanced questions. All questions were single-answer multiple choice, had an option for “I don't know,” and were presented in order of difficulty. Students were encouraged to select this option instead of guessing at an answer to help place them in the correct PAL group based on their true skill level. Approximately half of the questions required the students to listen to an audio file, one quarter of the questions required the students to read and interpret text, and the advanced questions required the students to watch a video encounter of two native Spanish speakers holding a rapid conversation. Students were not provided with an answer key after completing the test. After the completion of the medical Spanish portion of Clinical Correlates, students took a course post-test that was identical to the pre-test.
Out of 20.0 possible points, mean scores improved from 9.5 on the course pre-test to 15.0. Median scores improved from 9.0 to 15.0, and the lowest test score improved from 1.0 to 7.0. The difference in mean and median scores from pre-test to post-test is statistically significant with
p<.001, using a paired
t-test (
Figure 1). Student performance on each group of questions (based on difficulty) also improved from pre-test to post-test (
Table 1). The “I don't know” answer choice was selected 1,111 times in total on the course pre-test, but only 360 times on the course post-test. The number of times that “I don't know” was selected was significantly different for all of the questions from pre-test to post-test, with the exception of questions 2 and 4, using a sign test (
Figure 2).
We were not surprised by the improvement in test scores of students who started as very basic Spanish speakers, as the course assessment seems to be an accurate reflection of the information taught throughout the class. Students who had pre-existing Spanish experience improved approximately one level based on the course pre-test to post-test (e.g., from low-intermediate to high-intermediate), which we consider a success. At this time, however, we are unable to determine the students' true skill level in the real world. Regardless, the authors believe that the development of a tiered structure of difficulty for the course content would be beneficial to all students.
The number of times the “I don't know” answer was selected decreased dramatically and significantly from the course pre-test to the course post-test. Although the correct answer was not always selected, this suggests that students felt more confident in their abilities after the course. Anecdotally, we witnessed this confidence during the encounters with the simulated clients, where the students were willing to give their best attempt at communicating in Spanish.
One limitation of the post-test was that the same questions were used as in the pre-test, but this influence was reduced by the “I don't know” answer option, the inability of students to view the answer key after taking the pre-test, and the amount of time (and other coursework in the curriculum) that elapsed between the two tests. Four students completed the course pre-test but not the post-test. This emphasizes the need to attach a grade to performance in the course, rather than relying simply on attendance.
Year 2: Successes and Challenges
Content Delivery
The introduction of the material on a LMS was relatively painless and worry-free due to the skills of the technology support staff. Barring a few intermittent technical errors, students were able to access all course material and to take the assigned tests. Any parties interested in taking a similar approach would be well advised to first ascertain the technology and support staff at their disposal.
The students who benefitted most from this approach were the novice speakers, who could watch the lectures as many times as needed to learn the material. An unintended benefit of digitizing the lessons was that we have been able to utilize them for other purposes. For example, we distributed the digital lectures before a continuing education workshop on medical Spanish to give those participants the opportunity to prepare in advance.
Simulated Client Interactions
The authors of this paper are of the opinion that the introduction of simulated clients to the medical Spanish course had the greatest learning impact on the students. It seemed to us, witnessing the interactions firsthand, that many students had a “wake-up call” to the practicality of the course material. In addition to the opportunity to practice the Spanish language, the simulated client interactions introduced students to different cultural norms. Student feedback on the encounters was also quite positive: “The client simulations were a bit nerve-wracking, but they ended up being very enjoyable and helpful in practicing communication with all types of clients”; “The client communications exercise in Spanish was a great way to top off the course! It felt very real to me and the actors did a great job”; and “I also really enjoyed role playing where we communicated with Spanish speakers.”
Peer-Assisted Learning
No changes were made in the second year to the PAL portion of the class. We continued to recruit fluent Spanish-speaking first-year DVM students to serve as peer tutors, each group was still composed of approximately eight students, and student engagement continued to be variable. We hope, however, that students who struggled during the simulated client scenarios will see the real-world applicability of the material and will consider learning it on their own time. Furthermore, we anticipate that the engagement of future classes will improve as they hear about the simulated client scenarios through word of mouth.