LEAP MOTION: THE POWER IS ON YOUR HANDS
This small device lets you interact with the computer by recognizing the movements from your hands and fingers, it can convert tri-dimensional gestures into instructions to create a seemingly touchless interface. Let me show a little more about this special device… Hands up!
HOW LEAP MOTION WORKS
The Leap Motion device provides a gestural control system that can be connected to any computer by USB. One of the surprising aspects of Leap Motion is its size, the first versions had a similar size to a Microsoft Kinect but the commercial version is only 80mm x 11.25mm x 30 mm. Don’t be deceived by its size however, inside the device there are two cameras and three infrared led’s that lets the device detect hands and fingers placed at a distance of up to one meter away, with a precision of approximately 0.01mm. It is also capable also of detecting pencils or other tools in you have in your hands.
- The Leap Motion can be easily transported and only needs to be plugged into a USB connector, it’s also supported most of the major operating systems.
- It can be bought for only $80. This makes it a very inexpensive device for such an experience.
- The Leap Motion gives you aunique user experience. You can’t control your app the same way with your mouse, keyboard, gamepad or touchpad as you can do directly with your hands and Leap Motion.
- The device has great precision. It can be improved, but 0.01mm is a more than sufficient for a tridimensional gesture control device.
- Despite the good precision from the device, sometimes it shows a poor detection for vertical hand positions. If anyone is moving near you Leap Motion can detect these movements as well, which leads to interference.
- A few hours of use are enough to heat the device and producing lag in movement detection.
Unlike when we use traditional peripherals as joysticks, gamepads, the mouse or the keyboard, when using Leap Motion we don’t have a specific number of buttons to assign to our game actions. We have a lot of data from the device and our code must be able to translate all the relevant information (such as gestures, speed, initial and final finger positions, etc) into game actions.
This point forces us to create a simple, intuitive and versatile user interactions to explain to the user all the gestures and movements necessary to control the game.
Sometimes the control of our app can be tedious and unclear to the user, so you’ll need to consider giving the user a short and clarifying tutorial demonstrating the different ways to control all the aspects of the game. Additionally, after the user has been playing for a long time, you must keep aware that excessive use of gestures can produce muscle fatigue to the user, so we must study very well the UI in our design.
In order to improve the user experience the ideal scenario will require us to design a user interface fully integrated with Leap Motion. It will be tedious if the user needs to sometimes use the keyboard (to introduce a username or a form data) or even the mouse, so need a UI that can work with the device in order to create a Natural User Interface.
The Leap Motion has his own store where you will find a lot of games and apps that demonstrate the different utilities of the device. You can access to the Leap Motion Store through its own app or by using their website: Leap Motion
Let’s walk you through a few examples you’ll find in the Leap Motion Store:
Kyoto is essentially a beautiful experience. A peaceful place where you can gently interact with the base interactions of the Leap Motion. A relaxing experience for those frantic days. With Sculpting you’ll be able to create 3d sculptures with your hands. Sculpting offers a variety of tools, brushes, materials and colors for your creations, even clever spin and mirror functions. Once your model is finished you can export it in to different 3d format files.
You will obtain better results using a big size screen but you’ll find Sculpting is a very interesting app. You can take a look at it here:
From the hand of the famous script writer Stan Lee, Verticus will make you feel like a real superhero. You control the character with hand movements while you take part on this fast, exciting and fearless adventure.
This game show us how simple gesture controls can create an amazing and immersive game. You can hear Stan Lee talking about Verticus and Leap Motion at:
Although Form and Function 3D only lets you explore the hearts of three different animals (shark, salamander and cat), it’s a great example of how Leap Motion could be used in educational environments (See page 6, 1), to learn more about Form and Function 3D and see how it works, check it out here:
Virtual Piano for Beginners is another educational and really interactive example application. With this app, not only will you be able to play a virtual piano in the air but you will able also to take piano lessons for beginners.
Sometimes there are precision errors but it’s definitely a funny app for children and adults. This is a clear example of integration between a Leap Motion and the Oculus Rift VR headset, the control device can be mounted on Oculus so you can easily to develop a VR experience with free hands. But with Tran;section we also find an interesting experiment, it’s really a layered game (a game inside a game), playing with layers and 3D effects we will be immersed in a new game level, showing us a complex world using a simple development.
LEAP MOTION AND UNITY
Leap Motion have developed a Unity Plugin, so the integration of the device in Unity projects is very easy. First you must first download the plugin right here.
Once you have downloaded the file, you only need to unzip it and import the unity package into a new project. The package includes the core asset for Leap Motion and the VR asset for using Leap Motion in union with Oculus Rift. In order to make a more generic approach to the Unity integration we will cover only the core aspects of the package.
To start using your hands in your project is very easy, you simply need to place the HandController prefab in your scene (you might also need to adjust the position and the scale for the controller). When you hit play and put your hands over the Leap Motion, you will see a robotic hand that you will can control directly by moving your own hands.
There are different options you can setup in the controller, the package a lot of examples, from human to robotic ones, with a forearm or just the hand, you can also add your own models too. If you wish, you can also modify the physics for the hands on the controller and add a tool like a flashlight for example.
If you want the hand to interact with other objects, you just need to add a rigid body component to each hand, the hand will then be able to hit any objects in the scene (that have physics colliders and rigid bodies) (see above), that way you will be able to move, push and pull objects in the scene. You can also test also the HandControllerSandBox prefab, with this you will find a room already setup with colliders and a simple hand controller. In this scene you can try to push or launch the different objects to the walls to play with physics.
That’s not all you can do however, the package also adds the C# libraries for Leap Motion so you can access all the functions of the Leap Motion directly from your code, by adding the “using Leap Motion;” statement to the top of a script file, With this you can access the API to detect predetermined gestures like the circle, swipe or key tapping movements, or even add code new patterns to detect your own defined gestures. If your programming knowledge isn’t strong enough to develop your own gestures, you can take a look to the Unity Asset Store where you can find some packages that will help you in your creations. You can get access to the full API reference right here.
Anyway, the package contains a lot of examples demonstrating all the different possibilities of the device, with these examples you will learn how to use and develop for Leap Motion.
Leap Motion is a very nice and compact device that can be also mounted on an Oculus Rift, this lets us to control our app or our game by using only gestural controls. This feels like a more natural user interface which feels closer to the user.