At 3 years old, Deb Roy’s son is already among the most-filmed human beings in history. Nearly every waking moment of his life has been recorded by cameras and microphones installed in his home. The person responsible for this is his father, a professor and researcher at the M.I.T. Media Lab whose previous work involved building robots that could learn how to meaningfully converse. Curious about how real people went about it, Roy decided to make the longest home video in the history of parenting.
The grandly named Human Speechome Project will, Roy hopes, answer some of the longest-standing questions about language acquisition by capturing all of the tiny breakthroughs that make up the process: When a child first says a word, what is he looking at? What has just been said to him? Do the different ways his mother, father, and nanny speak to him have an effect? “The cradle of language is the home,” says Roy. “The social interaction and the physical interaction, that’s the fabric out of which language and communication arise.”
Three years of filming his son have produced a total of 90,000 hours of video and 140,000 hours of audio. Roy and his team are developing an arsenal of high-speed speech-transcription software and movement- and gesture-tracking software to sort through it all. Early this year, they will submit one of the first papers based on Speechome results about what Roy calls “microbursts”—small, previously unnoticed spurts in his son’s language acquisition that Roy believes might be the building blocks of the vocabulary explosion most 2-year-olds go through.
There are more-immediate applications as well. Roy has designed a “Speechome lamp,” a portable unit with the same fish-eye lens and microphone he installed in his own house, which has the potential to identify symptoms of autism far earlier than traditional doctor’s visits. He is also in talks with a corporate sponsor to design a Speechome lamp for the public—for parents who want to never, ever miss a moment.