Improving Internal Consistency in Conditional Probability Estimation With an Intelligent Tutoring System and Web-based Tutorials

Three web-based laboratory experiments explored the efficacy of three different web-based tutorials designed to improve performance on Bayesian conditional probability estimation problems. In each experiment, participants estimated the probability of two events, and two conditional probabilities P(A|B) and P(B|A). Problems reflected five distinct relationships between two sets: identical sets, mutually exclusive sets, subsets, overlapping sets, and independent sets. Performance was measured against two benchmarks: internal inconsistency, a type of fallacy, and semantic coherence, a constellation of estimates of P(A), P(B), P(A|B), and P(B|A) that are consistent with the relationship among sets presented in the problem statement. As predicted by Fuzzy-Trace Theory, in all three experiments, problems depicting identical sets, mutually exclusive sets, and independent sets yielded superior performance with respect to inconsistency and semantic coherence than problems depicting subsets and overlapping sets. In Experiment 1, a web-based tutorial teaching the logic of the 2x2 table reduced internal inconsistency for overlapping sets problems. In Experiment 2 a web-based tutorial using Euler diagrams was effective in reducing inconsistency and increasing semantic coherence for overlapping sets and subsets problems. Experiment 3 employed AutoTutor Lite, the first web-based Intelligent Tutoring System with two-way interactions with people in natural language (English). AutoTutor Lite is cross-platform enabled with talking animated agents that converse with learners using Latent Semantic Analysis to "understand" natural language. AutoTutor Lite elicits verbal responses from the learner through a textbox and encourages them to further elaborate their understanding. AutoTutor Lite tutorial significantly reduced internal inconsistency on overlapping sets and subsets problems.