About NJU Motion Lab
NJU Motion Lab is located at Nanjing University, Suzhou Campus. It is directed by Difeng Yu.
As technologies such as 3D content generation, virtual/augmented reality, robotics, and wearable devices continue to advance, we anticipate a growing shift towards human-computer interaction in 3D space. Such interactions may take the form of digital avatars and virtual objects in VR/AR, or of shape-changing displays and embodied agents like robots. At NJU Motion Lab, we study how humans and computing systems interact through motion in 3D space. More specifically, we focus on:
- Input: how humans express their intentions in 3D space through motion and how computing systems interpret them, to achieve better intention communication.
- Display: how computing systems use virtual and physical motion to assist humans in 3D space, so that humans can make better decisions and fulfill their intent.
To get a sense of how we turned those high-level concepts into concrete works, you are welcome to read the following section on a short tour of Difeng's previous research.
A short tour of Difeng's previous research
Before joining Nanjing University, Difeng has been focusing on supporting mid-air hand-based input for interacting with virtual objects in VR. Hands allow us to interact with physical objects in everyday life, but they face many challenges when interacting with virtual ones, including:
- Imprecision. Mid-air hand-based input is imprecise, making it difficult to select small, faraway, and occluded targets. To address this, we have introduced visualization techniques that help users search for and specify occluded targets ↗. We have also developed models that predict an intended target from movement features to automatically correct user input ↗, and an intelligent system that proactively suggests targets at the right moment, without requiring users to aim at them manually ↗.
- Fatiguing. Mid-air hand-based input is fatiguing, a phenomenon commonly referred to as the gorilla arm effect. To alleviate this, we introduced motions from other input modalities into the workflow of hands. We have combined eye gaze with hand-based input, filtering out unnecessary gaze signals to enable a smoother transition between the two modalities ↗. We have also combined lightweight on-body input with hand-based input ↗ and proposed Bayesian models for fusing multiple modalities ↗.
- Confined. Mid-air hand-based input is limited in movement range and degrees of freedom, making it difficult to handle out-of-reach targets and objects that require large rotations. To overcome these limits, we have explored movement mappings that, for example, map a small physical hand movement to a much larger range in virtual space, enabling superhuman interaction ↗. We have studied how to measure the learning difficulty of such techniques ↗ and developed new techniques that are easier to learn.
Please don’t get stuck in a problem-solving mindset when reading the summary above. Many of these are retrospective, and we often pursue random ideas that spark. More recently, we have been fascinated by the theories and concepts in motor learning, motor control, and sensorimotor decision-making. Based on these, we have, for instance, developed a method to derive interaction techniques from theories ↗, and prototyped a virtual body that can react faster than a human user ↗. Research is a complicated process, and a scattered idea may someday connect to a bigger picture. In any case, if you find the overall research direction or a particular idea interesting and want to join us and Motion together, I encourage you to read further about things to know before you apply.
Things to know before you apply
- I would encourage you to apply if you enjoy the type of research we do and you are highly motivated to define your own directions based on our existing work. Ideally, your core motivation is not to “get a paper out of this experience,” but to conduct research that both you and I are excited about.
- Since you will be working with me, in some cases for quite a long time, you probably want to know what kind of person I am at work. Similarly, I also want to know you beforehand. For this reason, candidates who have worked with me before applying for a later-stage position will be preferred.
- PhD and MSc applicants should refer to NJU Graduate Admissions and the School of Intelligent Science and Technology Graduate Admissions for relevant information on applications, including application tracks, entrance exams/interviews, qualifications, and timelines.
- Postdocs will need to pass an entrance interview, most likely administered by faculty members other than me. Please contact me directly for information about the available scholarships and other requirements.
- I am very happy to take undergraduate students; I was attracted to the field when I was an undergraduate myself. This opportunity is available to students from Nanjing University (Suzhou campus), who are encouraged to commit to at least six months of research training.
- For research assistant opportunities, please contact me directly since it is relevant to the available funding and space.
- Your research experience should match the level you are applying for. For PhD applicants, I don’t expect you to have top-tier publications already, but you do need to demonstrate your research skills. You will need some research experience to pass the entrance interviews (administered by multiple faculty members from our school).
- When emailing me about applications, please demonstrate that you have a reasonable understanding of this page. Thanks!