1. Zenaida Aguirre Munoz
  2. Associate Professor
  3. Effective Practices Integrating Engineering and Literacy in the Early Years
  4. Texas Tech University
  1. Delia Carrizales
  2. Research Associate
  3. Effective Practices Integrating Engineering and Literacy in the Early Years
  4. Texas Tech University
  1. Michelle Pantoya
  2. Professor
  3. Effective Practices Integrating Engineering and Literacy in the Early Years
  4. Texas Tech University
Public Discussion

Continue the discussion of this presentation on the Multiplex. Go to Multiplex

  • Icon for: Sean Smith

    Sean Smith

    Facilitator
    Senior Researcher and Managing Partner
    May 11, 2015 | 08:37 a.m.

    As I watched the video, I was fascinated by the potential of students’ academic conversations. Can the project share some examples of these conversations, either in video or in transcripts? Also, I’m interested in what would sustain the level of conversation beyond the treatment phase, which is another way of asking why conversations diminished during the second baseline phase.

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 11, 2015 | 10:39 p.m.

    great questions. we can share some examples I will post some examples later this week. The combination of inquiry-based learning activities and specific conversation routines teachers implemented during the treatment phase were the catalysts to the conversations students had amongst each other. Absent both of those components, students were not motivated to share their thinking to others. Older students might continue after baseline period. We believe young students needed additional opportunities organized by the teacher to make it more automatic for 5 and 6 year olds.

  • Icon for: Sean Smith

    Sean Smith

    Facilitator
    Senior Researcher and Managing Partner
    May 12, 2015 | 08:50 a.m.

    That’s really interesting. Thank you for the follow-up. I wonder how long it would take for the culture to “stick,” for both the teachers and the students. At what point would it just become “normal”? I see from your response below that teachers “hated” having to go back.

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 13, 2015 | 12:39 a.m.

    Sean, that is a great question we will be examining transcripts more carefully for evidence of this impact. Interviews with teachers will also help us determine how their instruction changed after the second baseline. Do they report adding the conversation piece and inquiry learning again and in what ways. Future data collection could take another snap shot of instruction 4-6 weeks after intervention to see if their reports of engaging students in more conversations are observed by the research team.

  • Icon for: Davida Fischman

    Davida Fischman

    Facilitator
    Professor
    May 11, 2015 | 12:13 p.m.

    Great goal! Academic discussion is a big part of many of our projects also, and I am intrigued by your focus on this. Sustainability is a major challenge for many of us, and I’d like to hear your take on it. Aspects that we tend to look at include the length of the treatment as well as the cultural changes and technical systems put into place by the partnership to extend the gains beyond the project. I would love it if you could share some thoughts on the issue.

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 15, 2015 | 02:08 p.m.

    I just realized I didn’t fully address your question. Despite the fact that we have very close relationships with the district, when there is a change in administration sometimes it feels like we need to start over gain. We provide all the training materials to the district, the teachers, coaches and administration hoping that at least teachers and coaches can continue modeling for other teachers even if a new administration decides they want to go in another direction. We are looking for models for a more systematic approach to this for our next phase of work. Some of the projects featured in this showcase are very useful for thinking about this so I will following up with some. This showcase has been amazing I am so glad we decided to participate.

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 11, 2015 | 10:45 p.m.

    We trained teachers on specific routines that they articulated really buying into. They expressed ‘hating’ to have to go back to instruction without conversation routines. They all commented “do we really have to back to our old way of teaching?” “…this is powerful …my kids love it and I could see their thinking better now.”

    Of course, buy is from all teachers is very difficult. From my past experiences, getting the principle to buy in and make teachers accountable helps motivate more at least try the approach. Once they try, they see almost immediately that their shyer and even more challenging students engage productively.

  • Icon for: Sean Smith

    Sean Smith

    Facilitator
    Senior Researcher and Managing Partner
    May 12, 2015 | 08:53 a.m.

    There’s a model of teacher change (I can’t put my finger on the reference at the moment) that suggests a change in attitudes follows a change in instruction, rather than vice-versa. It sounds like that’s what you’re seeing…once teachers see some results, their attitudes about the new strategy change.

  • Icon for: Janet Kolodner

    Janet Kolodner

    Facilitator
    Regents' Professor Emerita
    May 13, 2015 | 10:23 a.m.

    Zenaida,

    Please make sure you assure the teachers that they can continue to engage their kids in this kind of discussion no matter what other learning activities they are engaging them in. I wonder how you can help them learn how to do that.

    Janet

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 13, 2015 | 02:53 p.m.

    You bet we did. In the second round of piloting, we told them they should use the strategies and materials they would normally use, if that then included academic conversation routines, we allowed it. Fortunately we had included a couple of teachers who were not trained yet on these routines so we were able to compare at baseline the added value of these routines to ‘normal’ practice. That is next year’s presentation. The routines that are provided are easily adapted to any content area. These teachers were able to modify during reading instruction and social studies.

    But this raises a very relevant ethical dilemma as researchers. If we know a strategy is very beneficial how can we build into our design ways to provide all students (and teachers) this opportunity. Thank you for raising that it will be most relevant when we try to scale this up.

  • Icon for: Sean Smith

    Sean Smith

    Facilitator
    Senior Researcher and Managing Partner
    May 13, 2015 | 07:39 p.m.

    Zenaida, I have heard of drug trials that were ended early because the evidence of efficacy was so compelling and the decision was made to give the drug to the control group. (Of course, in some instances the evidence was not as compelling as originally thought.) I can’t think of an education intervention that has a similar story, but I think that’s the dilemma you’re describing. One design I’ve seen used successfully (and I’m guessing you’re aware of) is to randomly assign participants to treatment and control groups and then give the treatment to the control group after the study is over. That way, everyone gets the benefit, just some a little later than others.

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 15, 2015 | 02:15 p.m.

    Sean,
    thanks for calling out for sounding very arrogant in my reply. It wasn’t my intent. I wasn’t specifically thinking about our impact, but the perspective from district administrators from very high need areas who tell, me “if you think your method is better, I don’t want to wait..please train all my teachers.” When working in schools where achievement is low and they are in or in danger of being identifies as in need of program improvement that is a common response. I have used the design you describe, but it is still concerning for me when I see that they need a lot of support.

  • Icon for: Arthur Lopez

    Arthur Lopez

    Facilitator
    Computer Science Teacher
    May 11, 2015 | 10:56 p.m.

    Very interesting; I am a Computer Science and bilingual teacher that works in an inner city school whose student population is 90% on free or reduced lunch programs and 70% have either been previously identified or are currently English Language Learners. Many of my students struggle with reading comprehension and understanding of technical and abstract terminology required in Computer Science courses. I would have loved to have seen video excerpts of students engaging in conversation and utilizing the engineering STEM materials. I am also interested to know if follow up studies are being considered to continue tracking these students as they progress through subsequent grades? And would STEM engineering materials be provided at appropriate grade levels as students continue to learn about STEM engineering design in 1st, 2nd or 3rd grades? Have you consider the evidence that you can provide that determines the impact of the STEM engineering materials as children get older? I also wonder if the ELL students would rapidly improve their language skills in both their primary and English languages? I really think it is great that you are doing research at such an early and critical childhood stage!

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 13, 2015 | 12:34 a.m.

    Thank you Arthur for your comments and questions. We wanted to show video, but our IRB does not allow it. We will be using transcripts of conversations to illustrate the progression they go thru. For these young students, we started with pairs and asking them to repeat what the other said to train them to listen to their partner’s contribution. Then we added one conversation skill at a time. Our project is an small research and development grant, so we don’t have the resources to follow them, that will be our next larger grant. Our main goal was to develop and test the literacy-based resources and activities and to test the feasibility of training teachers on this approach. Yes, we started at the youngest level and will develop materials each year for subsequent grades. Our design does not follow these students. We will be collecting additional assessments in grades 1 and 2 which will allow us to answer the impact as children get older, but this is cross sectional not longitudinal. Yes, we do hope to eventually test more directly if the set of materials helps accelerate English proficiency. An examination of students’ language use will provide us some clues now, but more direct investigations of this questions would require more classrooms and additional resources.

  • Icon for: Arthur Lopez

    Arthur Lopez

    Facilitator
    Computer Science Teacher
    May 13, 2015 | 11:24 a.m.

    Thank you for the thoughtful and very clear reply, Zenaida. I do really appreciate it. I think your work is very important as it focuses on exposing students at a very young age to STEM; reading other studies I know that this has an impact and increases the possibility of these students becoming interested and pursuing STEM degrees. Wonderful project!

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 13, 2015 | 02:54 p.m.

    Thank you Arthur!

  • Icon for: Janet Kolodner

    Janet Kolodner

    Facilitator
    Regents' Professor Emerita
    May 12, 2015 | 02:24 p.m.

    Very interesting project. And I think the questions asking for examples, about the teacher change model, and about making the culture stick (which is really what you need to happen and learn about for real impact) are right on. I look forward to more detailed answers on those things.

    As well, I wonder what was the difference between the treatment and other group. Did the other group use the same materials but not engage in the same discussions? Or something else? When I know, i will ask some additional questions and make some additional comments.

    I wonder, too, how you extended what the EIE folks learned while developing their curriculum. I know EIE a bit, though I don’t know much about the evaluations they did while designing it or after, and I don’t know what their teacher instructions say about promoting discussion.

    Finally, your conclusions are nice — they tell us that it is worth continuing to do design activities with those who do not yet know how to have an idea-based kind of discussion and that EIE curriculum materials are good for promoting such discussion when used a certain way. But I wonder if you are finding out some other things: (1) Are your results just for EIE and the particular way your teachers are using it, or do they generalize beyond that? And, if so, how do they generalize? (2) How, exactly, are your teachers using EIE? That is, I want a lot more detail about the treatment. (3) Finally, there is literature out there (some I wrote) about using design activities well to foster learning. What might you add to that literature? And how did that literature help you? I see you used brain literature, but I don’t see how you used cognitive literature or design-based learning literature to guide your work, and I think bringing those literatures in will allow you to get to more specificity about how to foster the kinds of discussion you want and also help you generalize from your results beyond EIE.

    One more thing, there is some literature on creating culture in the classroom — not so much with quantitative results, but ideas that come from what we know about cognition (some I wrote here also). That should help as you move forward from finding it is worth using design activities to foster ideas-based discussion towards figuring out how to sustain the culture being created.

    Janet

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 13, 2015 | 01:52 a.m.

    Thank you for your great comments and great questions Janet. Our project was focused on development of the intervention, not instructional change. We needed to train teachers on the use of the units and in the conversation routines in order to test the degree of feasibility for young elementary students. I absolutely agree that real impact on teacher practice would require an examination of their practice after the professional development. I am also aware of the literature that demonstrates that teacher beliefs about learning and instruction are associated with the degree of change in practice and therefore effective PD would target these beliefs and examine their practice more systematically. We did examine instruction for degree of fidelity to the treatment, but the outcome measures were focused on student engagement. (In case you are curious our fidelity was high and reliability on fidelity coding was very strong-exact agreement and Kappa.)

    In terms of design, for a single subject design, a separate control group is not necessary. The students act as their own control, which is why we included 2 baseline periods. This design is rigorous and conforms to the What Works Clearinghouse standards. But our next step is to conduct a group design where we would have an intervention group and a control. So your question is timely and is the next step. Since there was not enough past research to guide reasonable hypothesis on the direction of student outcomes with respect to our combination of activities with ELLs, a single subject design made the most sense to start. If all goes well we will be reporting on results from a group design this time next year.

    With respect to our adaptations to EiE, yes we did modify! But all 4 lessons were implemented. We reduced the length of the story context (lesson 1), in this iteration, our text modification may have led to the reduction in student engagement during the reading of the story context. We used the hand pollinator unit which included a play, so it could also be an artifact of the genre for this particular unit. We took what we learned from this initial test and made guidelines for text modifications and have implemented it again. We are looking at the data and will know soon if our text modifications were better this time around in terms of keeping students engaged. Other modifications were to the worksheets students used to record what they learned about the materials in lesson 2 and 3. We lessoned the linguistic load to be appropriate for emergent readers. Lesson 4 was implemented as designed by EiE. We also added assessments to their existing assessments, a card sort that targeted, understanding of technology, the design cycle and the science concepts. We also implemented a think a loud protocol while they are doing the card sort so we can understand their thinking process while engaged in that task. We also embedded academic conversation routines in each of the 4 EiE lessons as well as the technology in a bag ‘pre lesson’.

    In terms of your other questions: 1) Are your results just for EIE and the particular way your teachers are using it, or do they generalize beyond that? And, if so, how do they generalize?
    Since we were able to find the same trend across types of students (low and high ELLs, and low, moderate, high ability students), and across classrooms (4; 2 mainstream contexts and 2 dual language immersion) we achieved a critical standard for a single subject design—replication across settings and individuals. Thus, we can generalize across to these two types of classrooms contexts. That is, we would expect that mainstream and dual language teachers who engage students in academic conversations while engaging in hands-on science and engineering design tasks (EiE includes both) would have similar results for their ELLs.

    (2) How, exactly, are your teachers using EIE? That is, I want a lot more detail about the treatment.
    See response above regarding EiE use. I hope that is enough detail.
    (3) Finally, there is literature out there (some I wrote) about using design activities well to foster learning. What might you add to that literature? And how did that literature help you?
    Janet, your work very much aligns with ours. We use the same research base in terms of what we know from the learning sciences. The brain stuff I like because of the great visual it provides and emphasizes the need to highlight the importance of scaffolding disciplinary discourse—which you also use. We also use systemic functional linguistics to frame the discourse work because it provides very specific language targets for teachers and students. Since the focus is on ELLs, we find that language framework useful when examining discourse patterns and in providing teachers with cues for what to listen for when they monitor student conversations. Most of the work in academic conversations is qualitative and corpus based, we hope to add to the field by confirming what qualitative (corpus-based) researchers (applied linguists) have been claiming for at least 15 years (much longer for learning sciences researchers)—in short, discourse matters. The nature and quality of students’ disciplinary discourse matters in both the development of understanding and in the ultimate attainment of targeted concepts. But we hope to provide implementable routines teachers can be trained to use and modify to fit the needs of their students.
    I will definitely look for more of your work to help us shape the classroom culture from a cognitive perspective within an engineering education context. The work we site is more general work in cognition. Thank you for that tip!

  • Icon for: Janet Kolodner

    Janet Kolodner

    Facilitator
    Regents' Professor Emerita
    May 13, 2015 | 10:25 a.m.

    So glad you did not have a control group. It seemed way too early for that, but the way you described the research, I thought you were doing that. Phew!!!

    More later on the rest.

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 13, 2015 | 02:55 p.m.

    looking forward to more of your great questions and comments!

  • Icon for: Michel DeGraff

    Michel DeGraff

    Professor, MIT Linguistics / Principal Investigator, NSF Project
    May 14, 2015 | 04:04 p.m.

    Congratulations for this important work! As a linguist engaged in using local languages for transforming the teaching of STEM in Haiti from rote memorization to active learning, I am fascinated by your research to improve education for underserved populations. I couldn’t agree more with you that, indeed, “discourse matters.” So language choice, too, matters: Students cannot effectively explain their thinking in a language that they do not speak fluently. And as Janet Kolodner points out, classroom culture too matters. In the Haiti case, we’ve seen first-hand the importance of choosing the native language of Kreyòl for teaching STEM instead of French which, for most students and teachers, are a (more or less distant) second language. Once students (and teachers!) are free to use their native language, and encouraged to do so by technology in that language, they become so much more effective in expressing and sharing their thought processes in academic conversations. I too would love to see samples of the conversations. I would also like to learn more about the native-language background of your emergent bilinguals and what role, if any, their native language played in their process of engaging in these academic conversations that you’re studying in your project. Thank you!

  • Icon for: Zenaida Aguirre Munoz

    Zenaida Aguirre Munoz

    Lead Presenter
    Associate Professor
    May 15, 2015 | 02:16 p.m.

    Thanks Michel,
    The students in the Dual language instruction classrooms were Spanish speaking immigrants from Mexico. In one classroom model, 90% of instruction is delivered in Spanish, in the second model 50% of instruction is in Spanish.

  • Icon for: Michelle Pantoya

    Michelle Pantoya

    Co-Presenter
    Professor
    May 15, 2015 | 01:42 p.m.

    Thank you for this comment Michel. We will work on the cultural component more as we move forward. Your point about the role of the student native language in their process of engagement is well taken.

  • Icon for: Deborah Kariuki

    Deborah Kariuki

    Computer Science Teacher
    May 15, 2015 | 09:43 p.m.

    Good start. This is as very interesting way of engaging students and getting them to communicate. How do you account for their level of reading as it would seem this would hinder understanding of the academic language where the words are not your simple daily conversation words? Also I wonder how fifteen days can actually give you data that is truly indicative of a change achieved. Nevertheless I see the potential in this work when the underlying issues reading level, school readiness etc. have been handled.

  • Further posting is closed as the showcase has ended.