The insight that powered Karpenko’s algorithms began, like so many other startup ideas, as a phD thesis at Stanford. This was 2010, and the iPhone 4 had come out: one of the first phones that could capture HD video. That sounded terrific, in theory, but cramming such a great video camera onto a handheld device meant that the videos themselves were often shaky to the point of being unwatchable. “They were all just crappy,” Karpenko says.
He knew that image stabilization was the answer, but the technologies of that time, which you’d find in Final Cut and myriad other video editing programs, were simply unworkable for smartphones. Why? Imagine a video clip, taken from a moving car. To even the juddering camera motion, image stabilization algorithms typically analyze a movie frame by frame, identifying image fragments common to each. By recording how those shared points jump around across frames, algorithms can then infer how the camera has been moving. By reverse engineering that motion data, software can recreate a new, steadier version of a film clip. Yet every step in that process requires processing muscle. That’s fine for a movie studio, which has massive computers that crank overnight to re-render a scene. It’s ridiculous for a smartphone.
Inspired by a demo in which he saw gyroscopes attached to cameras to de-blur their images, Karpenko had an aha moment: Smartphones didn’t have nearly enough power to replicate video-editing software, but they did have built-in gyroscopes. On a smartphone, instead of using power-hungry algorithms to model the camera’s movement, he could measure it directly. And he could funnel those measurements through a simpler algorithm that could map one frame to the next, giving the illusion that the camera was being held steady. He mocked up a simple demo, and filmed a dot on his wall, while making his hand shake. “The images in the test matched up almost exactly, and that’s when I knew this was doable,” Karpenko says.
