Project Category: Software
Join our presentation
About our project
In recent years, there has been a notable rise in the popularity of virtual reality (VR) headsets. The technology has become cheap and widely available allowing more people to own and regularly use VR headsets. There is one challenge in VR that has yet to find a satisfactory solution; text input. Physical keyboards remain the preferred and fastest way to input text in any environment as the VR solutions are too slow for anything other than typing a search query. Our group decided to develop a drop in replacement to the Oculus Quest keyboard solutions with the goal of maximizing skill transfer and comfort of a physical keyboard. Our solution was built in Unity and leverages the hand tracking capabilities of Quest devices. We designed our solution to use a flat surface to provide users with passive haptic feedback as they type, but the user can place their keyboard anywhere else they would like as well. Further details of the project and the team can be found below, and we hope you enjoy learning more about our project!
Meet our team members
Seb Schroh
Rylan Kettles
Brandon Khoo
Jack Glass
Zach Sims
Dylan Gordon
Details about our design
HOW OUR DESIGN ADDRESSES PRACTICAL ISSUES
- Encourages and enables the transfer of skill from a traditional keyboard to a virtual environment:
- One of the less desirable experiences with the current standard point and shoot method of VR text input is how foreign it feels for people who are used to using a traditional physical keyboard. Since our goal with this project was to provide a better text input solution than what already exists, we decided to design our solution in such a way that it acts and feels more like a traditional keyboard, thereby improving it’s ease of use.
- Since some people like to frequently change the position of their keyboard, we also designed our solution such that adjusting the keyboard position on the fly is easy to do. This decision was also influenced by our goal of having our keyboard be as easy and comfortable to use as a traditional physical keyboard. If people tend to use a physical keyboard in a certain way, we want to do our best to enable them to use our virtual keyboard in that same way as well.
- Doesn’t require any additional equipment:
- There are currently other virtual keyboards out there that work and feel similar to ours, however a number of them require additional cameras, gloves, or other peripherals in order to function. We wanted to create a virtual keyboard which can be used with an Oculus device right out of the box, and that requires minimal setup. We decided in the early stages of this project that if our solution requires tedious configuration or extra equipment, we are simply replacing one problem with another rather than actually improving on what already exists.
- The Oculus controllers are required in order to set the initial position and orientation of the keyboard, as well as reset the keyboard in a different position or orientation later on. While this means that we need the controllers for some initial configuration, the setup is incredibly simple; just put your controllers on either side of where you want the keyboard to be, and press both triggers. Considering that the controllers are only required for a very short period of time, and that the controllers come standard with all Oculus devices, we decided that our initial goal of requiring no additional equipment is still satisfied.
WHAT MAKES OUR DESIGN INNOVATIVE
- Solving old problems for a new medium with even newer technology:
- Virtual reality as we know it today is a very new technology, and by virtue of making use of some of today’s cutting-edge features of VR, we are creating something that few people are aware is even possible, and that even fewer people have attempted to create themselves. Hand tracking technology that does not require any peripherals besides the VR headset itself is an incredibly new development. In fact, when we first started discussing the idea of doing something like this as our capstone project, hand tracking was not yet an official feature for Oculus devices and could only be accessed by putting the device in developer mode and enabling beta features in the developer menu. It has been less than a year since the official release of Oculus’ hand tracking system and only a few years since the release of the first generation headset, and it has been incredibly exciting and interesting for us to be able to work on solving problems that are not even 5 years old with technology that is not even a year old.
- Brings benefits that are missing in other solutions:
- While there are a handful of other solutions to VR text input that are widely used, there are certainly various advantages and drawbacks to each one. While it would be ostentatious (as well as simply incorrect) to claim that our solution has no such drawbacks, we believe it certainly solves more problems than it creates. Point-and-shoot methods for text input, for example require that the user has both controllers in their hands at all times, creating an awkward experience for users who are using applications with hand tracking enabled. Drum/punch keyboards are other examples of different VR text input solutions, however they can be fatiguing to use after only a short period of time.
- Although it has been mentioned in other sections, one of the more innovative aspects of our design is that in addition to not requiring any controllers to be used for text input (controllers are only needed for about 20 seconds for initial set-up), there are no additional cameras or sensors required at all. Many other hand tracking keyboards that look and feel similar to ours also require the use of one or more camera, special gloves, or other equipment that ultimately takes away from the user experience of being able to type anywhere, anytime. We designed our solution to be innovative to the point of no longer requiring this additional hardware as well as allowing the user to have a vastly expanded freedom of movement by reducing set-up and repositioning effort.
WHAT MAKES OUR DESIGN SOLUTION EFFECTIVE
- Collision logic optimization:
- Implementing collision logic in game engines such as Unity is relatively straightforward in and of itself, and is often a minor implementation detail as the complexity will typically arise in determining how to respond to these collisions. For the case of this project, however, we allocated a large amount of effort to fine-tuning our collision logic. The reason that we did this is because typing on a keyboard requires much more precision than most other uses of collision systems. A slightly misaligned or incorrect hitbox in most video games or simulation software will cause some trouble here and there, but for virtual keyboards that require very precise input, hitboxes must be very accurate. Not only do hitboxes need to be accurate, the collision logic as a whole is the key to what makes a virtual keyboard feel how it feels. Fat-fingering and rapid double inputs are two of the most common errors that our solution is optimized to avoid, and it avoids these errors by implementing timers and checks on both the individual keys and fingertips. Without these optimizations, the raw hand tracking input to the keyboard will have drastically reduced accuracy as even a small mismatch between the tracked and actual position of a fingertip could result in multiple unintentional inputs in the span of a very short time.
- Has passive haptic feedback:
- One of the challenges that exist when designing a virtual keyboard compared to a physical keyboard is the need for physical feedback. Physical keyboards can easily provide this physical feedback, and all of them do to some extent or another. For virtual keyboards, however, a user will quickly notice how much this lack of feedback affects their typing ability. To mitigate this, we designed our keyboard to be placed on a flat surface such as a desk or a table. This way, the user will experience passive haptic feedback as their fingers come into contact with the surface. While there is no haptic feedback for differentiating between individual keys, the feedback from the surface gives the user a very good sense of depth when typing.
- While the keyboard was designed to be placed on a flat surface, it by no means needs to be. If a user would like to place the keyboard in empty physical space, they can rely on the visual feedback of the keyboard, which will be detailed further below. Placing the keyboard in empty space will cause the user to lose out on the haptic feedback that comes with placing it on a desk, however we still consider our solution to provide passive haptic feedback as our design assumes a flat surface is being used.
HOW WE VALIDATED OUR DESIGN SOLUTION
- We decided to validate our solution against 4 other text input solutions: Unity 3rd party, Rift punch, Rift point-and-shoot, and physical keyboard. The Unity 3rd party keyboard is a point-and-shoot style keyboard that can be found on the Unity asset store. The Rift point-and-shoot is the default style of keyboard on Oculus devices. The Rift punch keyboard is a larger punch-based keyboard, and the Physical keyboard is simply a physical keyboard that most of us use every day.
- Quantitative validation:
- We decided to quantitatively validate our solution against the other 4 solutions using two primary metrics; words per minute, and correction efficiency. Words per minute is one of the most telling metrics when it comes to typing accuracy, however words per minute is only a single measurement and cannot tell the whole story. To account for this, we decided to also measure correction efficiency as we believe that correction efficiency is a very valuable metric in addition to providing insights that words per minute does not account for. While the physical keyboard far outperforms all other solutions in terms of words per minute, we are happy to report that our keyboard is very comparable in both words per minute and correction efficiency to all other VR solutions. The aggregate data across all tests can be found below:
- Qualitative validation:
- Considering how instinctively we use keyboards today, we thought that it would be important to make qualitative comparisons between our solution and the others. We decided to rank in 4 categories for intuitiveness, usability, effort, and enjoyment on a scale of 1-5. Just as it did with our quantitative metrics, the physical keyboard had the highest overall score in every category. Our keyboard was second only to the physical keyboard in terms of both Intuitiveness and enjoyment, and was dead middle in usability and effort. The average scores across all keyboards and categories can be found below:
FEASIBILITY OF OUR DESIGN SOLUTION
- Financial feasibility:
- Financially, we believe our solution is very feasible as it requires no additional equipment to use. Additional to this, any VR developer who wishes to implement our design in their own project will be able to do it very easily, making it highly financially feasible in terms of developer cost as well.
- We do not consider the cost of VR headsets to be part of the financial cost of our keyboard, as a user would only be using our keyboard if they already have a compatible VR headset. That being said, the cost of VR headsets has been trending downwards and will likely continue to as the technology becomes more and more accessible. This reduction in cost will eventually make the cost of additional equipment or software a more important factor, as most people would be more willing to purchase a $25 peripheral for a $600 headset than a $25 peripheral for a $100 headset. The fact that our keyboard is free and easy to use will make it even more financially feasible in the future, even though it is already very financially feasible currently.
- Usage feasibility:
- The feasibility of our project is more mixed, as some aspects of our design make it more feasible, while others have a negative impact on the overall feasibility.
- The easy to use nature of our design paired with the simple setup and freedom of use are certainly positive factors when considering the feasibility of our design. There is very little learning involved as we designed our keyboard to be very easy to use for anybody who is familiar with traditional physical keyboards. Other aspects of the design such as modelling the user’s hands as they are tracked or lighting up keys as they are pressed also improve the overall feasibility of our keyboard for every day use.
- We are aware of some of the aspects of our design that affect the overall feasibility of the keyboard negatively, such as the lack of more detailed physical feedback, and occasional hiccups using the Oculus hand tracking system. We believe that our keyboard will perform significantly better as better as hand tracking technology improves with newer headsets and better hardware. Detailed physical feedback plays an important role in how easy a keyboard is to use and how comfortable it feels, however we believe that this may be an insurmountable barrier without requiring additional hardware of some sort.
- The feasibility of our project is more mixed, as some aspects of our design make it more feasible, while others have a negative impact on the overall feasibility.
Partners and mentors
We want to thank those who helped us with this project a reality. Our academic advisor Dr. Hu, our T.A. Manuel Lopez, and our industry representative Greg Lund. Without their invaluable guidance and constant support, we would not have been able to overcome many of the obstacles that we came across throughout this project.
