Qi Sun is expanding the AR/VR experience

Qi Sun, a joint professor in Computer Science and Engineering and the Center for Urban Science and Progress, has a mission: to create computer graphics and AR/VR experiences identical to the ways in which humans perceive in real life.

He has already made enormous strides. A former research scientist at software giant Adobe, he and his colleagues developed a technology that draws upon the phenomenon of saccades — natural, rapid eye movements that occur several times a second when we look at different points in our field of vision, such as when we scan the road ahead of us. During those minuscule shifts, our brains disregard visual input, leaving us unaware of the movements our eyes have just made. Sun and his fellow researchers realized that if a user was wearing a VR headset and the camera was rotated just slightly during saccades, they could simulate a vast open space for roaming — no matter how small the user’s actual physical space was. Called Dynamic Saccadic Redirected Walking Technology, it could one day be used not only in gaming but in any field that could benefit from truly immersive simulations, including architecture, education, and urban planning.

Sun is especially interested in how simulating human sensations can help make cities more livable and inform public policy. “If we can effectively model how people sense visual and auditory signals, for example, we can determine the best ways to mitigate light and noise pollution,” he says. “We can determine where to allow new buildings to be erected or where more green space is needed.”

Working in New York City not only provides Sun with a living urban lab, it also serves as something of a homecoming: in 2018 he earned his doctoral degree not far away, at Stony Brook University, where he garnered an IEEE Virtual Reality Best Dissertation Award.

NYU Tandon, he notes, will give him access to experts in fields like neuroscience, enabling the cross-disciplinary research he wants to undertake; allowing him to expand what can be accomplished in the realm of VR/AR; and giving him a chance to influence the next generation of computer science visionaries. “My students should not expect me to simply lecture to them,” he says. “This is not a passive field; you learn by doing and creating things of your own.”