Icon for: Noboru Matsuda


Human-Computer Interaction Institute, Carnegie Mellon University
Public Discussion
  • Icon for: Neil Plotnick

    Neil Plotnick

    May 11, 2015 | 02:18 p.m.

    How do you determine the competency level of the real students prior to assigning them to their tutorial roles? How does working with a Sim peer compare to students that work with a real peer? Does your software adapt to the skills gained by the users automatically or does an administrator of the system need to migrate users to higher levels?

  • Icon for: Noboru Matsuda

    Noboru Matsuda

    May 14, 2015 | 08:31 p.m.

    Hi Neil,

    Thanks for asking. We hypothesize that there would be a sweet spot for the prior knowledge (the readiness of teaching) that would most beneficial for learning and teaching, and understanding where would be an important research question. We do not have an answer just yet. A previous study where we compared learning by teaching and cognitive tutoring (aka learning by being tutored) showed that students who have low prior knowledge would learn more with cognitive tutoring. To answer where the sweet spot would be, we plan to extend our current software so that students can freely switch between learning by teaching and cognitive tutoring. That way, we would be able to measure the impact of prior knowledge on tutor learning.

    SimStudent is a genuine machine-learning agent that does learn skills when being tutored. SimStudent is a cognitive model of inductive learning, and a previous study showed that SimStudent has a cognitive fidelity in a sense that it makes same errors that students commonly make; e.g., subtracting 4 from both sides of 2x-4 = 10. SimStudent make such an error due to an induction error when “shallow” prior knowledge is given—e.g., perceiving ‘4’ in 2x-4 as a “number” instead of perceiving ‘-4’ as a term where such perception would work to, say, subtract 5 from 3x+5 = 9.

    We haven’t compared learning by teaching synthetic peer vs. real peer. An interesting future study!

  • Icon for: Jessica Hunt

    Jessica Hunt

    May 11, 2015 | 02:31 p.m.

    I noticed that the Sim peer asks varying questions to the teaching student (e.g., explain a step that was demonstrated; why disagree with a step the Sim peer suggested/“Why am I Wrong?”). What are the affordances of each kind of question? Do you find that varying types of questions asked by Sim peer are related with student learning (i.e. proficiency in solving equations)? How?

    Do you have any preliminary information on the nature/affordances of the adaptive scaffolding provided by “Mr. Williams”?

  • Icon for: Noboru Matsuda

    Noboru Matsuda

    May 14, 2015 | 08:40 p.m.

    Hi Jessica,

    Thanks for great questions!

    SimStudent attempts to solve a problem given by the student (who acts as a “tutor” for SimStudent). SimStudent asks “why” questions in the context of tutoring. It asks “why am I wrong?” when the student disagrees a step suggested by SimStudent. It asks the reason why the student suggested a particular step (e.g., “why did you divide both sides by 4”) when SimStudent got stuck and asked a help to the student; and the student suggested a next step.

    The theory says that when the tutee asks a constructive question, then the effect of tutor learning would increase. We are currently improving SimStudent’s ability to ask questions by integrating a natural language processing component to SimStudent so that SimStudent recognizes “good” and “bad” explanations.

    A previous study showed that when Mr. Williams provided metacognitive scaffolding on what problem should be taught then students tend to provide more appropriate problems to SimStudent, which results in increasing the change of passing quiz (i.e., increasing SimStudent’s proficiency) that further increases student’s learning.

  • Small default profile

    Jeremy circlcenter.org

    May 13, 2015 | 07:45 a.m.

    Do you think of this as an artificial sort of dialogue (real kids unlikely to be doing this with other kids) or a natural sort of dialogue (real kids might do it with other kids, it just doesn’t happen that often)? How does the naturalness or artificialness help or hinder learning?

  • Icon for: Noboru Matsuda

    Noboru Matsuda

    May 14, 2015 | 08:44 p.m.

    Hi Jermy,

    Nice question! In classroom studies, we have observed lots of students indeed believe SimStudent does understand their response hence they are engaged in typing responses. (at the same time, some students realize that SimStudent doesn’t really understand their input, which is true). We hypothesize that the rich dialogue between students and SimStudent would facilitate students’ conceptual understanding, which is a benefit of learning by teaching that does not happen in the context of cognitive tutoring (i.e., learning by being tutored).

  • Icon for: Stephanie Teasley

    Stephanie Teasley

    May 14, 2015 | 07:54 p.m.

    This video is essentially a demo of the system with too little information about the project as a whole, and the context for using such a system. Teachers watching will be left wondering, as I am, how this works in classrooms. One student at the end of your video says it was fun, but what do students do when they work with it? How do they feel about the “sim” aspect? Do you see gaming behavior as been found it other Intelligent Tutoring Systems? (e.g., Ryan Baker’s work.)

  • Icon for: Noboru Matsuda

    Noboru Matsuda

    May 14, 2015 | 08:51 p.m.

    Hi Stephanie,
    Thanks for asking!
    The demo you see is what students do. You, as a student, enter a problem; you quiz SimStudent; you answer SimStudent’s “why” questions. The interaction between you and SimStudent is the key component of learning by teaching.
    Yes, in the classroom studies, we occasionally observe students “gaming” the system—e.g., students enter silly response to SimStudent’s questions. We also observe suboptimal behaviors—e.g., students teach SimStudent only on “easy” problems that they can teach, which results in SimStudent’s poor learning (hence student’s poor learning as well).
    We have indeed started a collaboration with Ryan Baker and his colleagues to implement suboptimal behavior detector (including “gaming” the system) to provide the proactive scaffolding to students to enhance learning by teaching.

  • Further posting is closed as the showcase has ended.