NAV Navbar
Java C++ C# Unity Swift

Getting Started

//Please scroll below for code examples
//Please scroll below for code examples
//Please scroll below for code examples
//Please scroll below for code examples
//Please scroll below for code examples

Welcome

Welcome to the Mudra SDK! Use this documentation to create your own unique experiences based on the Mudra Inspire ™ device and Mudra API. This page is where you can get all the information you need to develop your unique experience.

Our API includes language bindings for Java (Android), C# (Windows, Unity), c++ (Windows) and Swift (iOS). In order to use Mudra in your application, all you need is to register a callback. First, follow the methods below and use auto-connect to instantiate Mudra. Use the methods that begin with set to register the callback you need. Afterwards, create the callback function in your code as detailed below.

Device Modes

Mode Description LED Indication
Pairing Flash green continuously alt text
Connected Flash green for 1 second alt text
Low Battery Quick burst flashing red alt text
Charging Flash blue continuously alt text

Android\iOS

Bluetooth should be turned on, if it is off then the Mudra Inspire application will prompt the user to turn it on.

Windows

Insert the included bluetooth receiver into the USB slot. See instructions below for installation and usage.

API functions

Access the API

Android

// gradle.properties
authToken='<Your Jitpack Token>'
// Project build.gradle
allprojects {
    repositories {

        google()
        jcenter()
        maven {
                url 'https://jitpack.io'
                credentials { username authToken }
        }
    }
}

// Java 8 support
compileOptions {
    sourceCompatibility 1.8
    targetCompatibility 1.8
}
// App build.gradle file:
implementation 'com.github.wearable-devices:MudraAndroidSDK:+'

// Bluetooth support
implementation 'no.nordicsemi.android:ble:2.1.1'
implementation 'no.nordicsemi.android.support.v18:scanner:1.4.0'

// Android manifest file:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

iOS

Windows

Installation

C++ Usage

Unity

When developing for Unity over Windows, please see installation instructions for Windows above. Following installation, see the file MudraUnityPlugin.unitypackage in the installation directory. Simply drag and drop this package into your Unity project’s assets folder. Go to the Hierarchy pane in your mudra editor, press create and select a Mudra manager. A new MudraManager Object will be created.

There are two ways to use the Mudra plugin:

(1) Using C# Unity scripts (requires coding). See Unity tab on the right for explanation. You can find the full example in the assets folder in the file named MudraUnityScriptExample.

(2) Using GUI on the MudraManager Unity object menu (no coding required). Using the MudraManager GUI, it is possible to enable\disable each of the features. For each feature, it is also possible to add callbacks.

IMPORTANT: Write the path of your calibration file in the GUI. Default file path is: "C:\Program Files\Wearable Devices\Mudra\Calibration.txt". In the Unity 'Poject Settings', set 'Write Permission' to 'External(SDCard)' instead of the default 'Internal' (in order to read the calibration txt file).

*Please make sure you are using the latest version of Unity and that the dongle is plugged in to your PC.

// Header files
#include "MudraEnvironment.h"
#include "MudraDevice.h"

Initialization and Connection

Android

Important: Please insert your authentication token in your app gradle.properties file in order to use our sdk. See our github example (see here).

Assuming the device is paired manually (for example - via the Android Bluetooth settings), there is no need for initial connection code. Initialization is exposed via the auto connect function.

Windows

Please call the Scan() function (note the indication in the console window) and press the button on you Mudra device. Once a Mudra device is discovered, the OnDeviceDiscovered() callback will be called. It is possible to check the name of the new discovered device and filter it if it is not your device. Please note that you need to declare on which hand the device is worn, see SetHand().

iOS

The iOS SDK supports developing both in Objective-C and Swift. The iOS API supports multiple Mudra devices.


// MudraDelegate SDK Code to be implemented
import Foundation

@objc public protocol MudraDelegate {

    @objc optional func onReadyForScan()

    @objc optional func onMudraDeviceDiscovered(_ device: MudraDevice)

    @objc optional func onMudraPeripheralDiscovered(_ device: CBPeripheral)

    @objc optional func onMudraDeviceConnected(_ device: MudraDevice)

    @objc optional func onMudraDeviceDisconnected(_ device: MudraDevice)

    @objc optional func onBatteryLevelChanged(_ device: MudraDevice)

    @objc optional func onScanFinished()
}


// User Code
import UIKit
import Mudra

class ViewController: UIViewController , MudraDelegate {

    override func viewDidLoad() {

        super.viewDidLoad()
        Mudra.shared.delegate = self

    }

    // A callback which is called once Mudra finishes initalizing
    // bluetooth and is ready for scan
    func onReadyForScan() {
        Mudra.shared.scan()
    }

    // Instead you can also run it but check before that Mudra is ready
    func someFunction() {
        if Mudra.shared.isReadyForScan {
                   Mudra.shared.scan()
        }
    }

    func onScanFinished() {

    }

    func onMudraDeviceDiscovered(_ device: MudraDevice) {
        print("Device \(String(describing: device.name)) discovered")

        // You may save this device for later use
    }

    func onMudraDeviceConnected(_ device: MudraDevice) {

    // Add your callbacks here

        // Note - Callbacks are called on a different ble thread, 
        // in order to update the GUI see below

        DispatchQueue.main.async {
        // Update the GUI here
        }

    }

    // In the AppDelegate.swift
    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {

        // Set your licenses (sent by Wearable Devices for expanded features)
        Mudra.setLicense(Feature.RawData, "--------") 
        Mudra.setLicense(Feature.DoubleTap, "--------")

        // Set Logging
        Mudra.setLoggingSeverity(.Debug)

        return true
    }

Logging

It is possible to set the logging severity by the following levels: Debug, Info, Warning and Error. You can set the logging severity using the SetLoggingSeverity(). It is also possible to use your own function for logging using the SetOnLoggingMessageCallback() function.

// Assuming the device is paired manually.
import MudraAndroidSDK.Mudra;
Mudra mudra = Mudra.autoConnectPaired(getContext());
using namespace Mudra::SDK::Windows;
using namespace Mudra::Computation;

MudraEnvironment mudra;

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{   
    if (device->GetName() != "MY_DEVICE_NAME")
                return;

    // Insert all your device's callbacks here. 

    // You can connect your device on discovery like presented here, or save your device and connect it later. 
    device->SetHand(HandType::Left);
    mudra.Connect(device);
}

void OnLoggingMessageCallback(const string &msg)
{
    // Do something with msg
}

int main()
{
    // Logging
    mudra.SetLoggingSeverity(Logger::Severity::Debug);
    mudra.SetOnLoggingMessageCallback(OnLoggingMessageCallback);

    // Connection
    mudra.Scan(OnDeviceDiscovered);

    // Calibration - if you do not use this file a default classifier\regressor will be used (it is highly recommended to use your own calbration file)
    mudra.SetCalibration("<YOUR_PROJECT_PATH>\\Calibration.txt");

    // Do something here (set infinite loop here if you would like to only listen to callbacks).
    // while(1) {...};
}
using Mudra.SDK.Windows.DotNetFramework;

static void OnMudraDeviceDiscovered(MudraDevice device)
{   
    if (device.Name() != "MY_DEVICE_NAME")
                return;

    // Insert all your device's callbacks here. 

    // You can connect your device on discovery like presented here, or save your device and connect it later. 
    device.SetHand(HandType.Left);
    MudraEnvironment.Connect(device);
}

static void OnLoggingMessageCallback(string &msg)
{
    // Do something with msg
}

static void main(string[] args)
{
    // Logging
    MudraEnvironment.SetLoggingSeverity(Logger::Severity::Debug);
    MudraEnvironment.SetOnLoggingMessageCallback(OnLoggingMessageCallback);

    // Connection
    MudraEnvironment.Scan(OnMudraDeviceDiscovered);
    MudraEnvironment.SetCalibration(HandType.Left, "<YOUR_PROJECT_PATH>\\Calibration.txt");

    // Do something here (set infinite loop here if you woould like to only listen to callbacks).

    // Close connection
    MudraEnvironment.Close();
}
using UnityEngine;
using Mudra.Unity;

public class TestMudraUnityPlugin : MonoBehaviour
{   
    void Start()
    {

    }
}

Fingertip Pressure

Functionality for estimating the amount of fingertip pressure. Returned values range between 0 and 1 and callbacks arrive at a rate of approximately 60fps.

Mudra.onFingertipPressureReady onFingertipPressureReady = new Mudra.onFingertipPressureReady() {
    @Override
    public void run(float v) {
        // Do something with v (this value is between 0 and 1) 
    }
};

mudra.setOnFingertipPressureReady(onFingertipPressureReady); // Enable 
mudra.setOnFingertipPressureReady(null); // Disable
void OnProportionalReady(float proportional)
{
    cout << "\nProportionalfound = " << proportional;
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnProportionalReady(OnProportionalReady);
}
static void OnProportionalReady(float proportional)
{
    Console.Write("\nProportionalfound = {0}", proportional);
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetOnProportionalReady(OnProportionalReady);
}
private void UpdateFingerTipPressure()
{
    float? fingerTipPressure = Plugin.Instance.GetLastFingerTipPressure();

    if (fingerTipPressure != null)
    {
        print($"FingerTipPressure {fingerTipPressure}");
    }
}

void Update()
{
    UpdateFingerTipPressure();
    //...
}
device.setOnProportionalReady {(proportional) in print( "OnProportionalReady \(proportional)") }

Gesture Recognition

Functionality for predicting six gesture types: thumb, index, tap, twist, double index and double middle tap. Please check out the Mudra app for more info and example usage.

Mudra.OnGestureReady onGestureReady = new Mudra.OnGestureReady() {
    @Override
    public void run(Mudra.GestureType gestureType) {
        switch (gestureType) {
        case Tap:
            // Do something here
            break;
        case Index:
            // Do something here
            break;
        case Thumb:
            // Do something here
            break;
        }
    }
};

mudra.setOnGestureReady(onGestureReady); // Enable 
mudra.setOnGestureReady(null); // Disable
void OnGestureReady(GestureType gestureType)
{
    cout << "\nGesturefound = " << MudraEnvironment::ToString(gestureType);
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnGestureReady(OnGestureReady);
}
static void OnGestureReady(GestureType gesture)
{
    Console.Write("\OnGestureReady = {0}", gesture);
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetOnGestureReady(OnGestureReady);
}
private void UpdateGesture()
{
    GestureType? gesture = Plugin.Instance.GetLastGesture();
    if (gesture != null)
    {
        print($"Gesture {gesture.ToString()}");
    }
}

void Update()
{
    UpdateGesture();
    //...
}
device.setOnGestureReady { (gesture) in print("OnGestureReady \(self.gesture2String(gesture: gesture))") }

Air Mouse

Functionality for exposing an air mouse based on IMU quaternions. The Mudra air mouse projects raw quaternion values onto a 2d plane, which can be used to move a cursor.

int mScreenWidth, mScreenHeight; // Size of screen in pixels

Mudra.OnAirMousePositionChanged onAirMousePositionChanged = new Mudra.OnAirMousePositionChanged() {
        @Override
        public void run(float[] floats) {

            float airMousePosX += floats[0] * mScreenWidth * HSPEED;
            float airMousePosY += floats[1] * mScreenHeight * VSPEED;
            float airMousePosX = Clamp(mAirMousePosX, 0, mScreenWidth);
            float airMousePosY = Clamp(mAirMousePosY, 0, mScreenHeight);
        }
};

mudra.setOnAirMousePositionChanged(onAirMousePositionChanged); // Enable 
mudra.setOnAirMousePositionChanged(null); // Disable
void OnAirMousePositionChanged(const vector<float>& position)
{
    cout << "\nX=" << position[0] << " ,Y=" << position[1];
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnAirMousePositionChanged(OnAirMousePositionChanged);
}
static void OnAirMousePositionChanged(float x, float y)
{
    Console.Write("\OnAirMousePositionChanged = {0}, {1}", x, y);
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetOnAirMousePositionChanged(OnAirMousePositionChanged);
}
private void UpdateAirMousePositionChange()
{
    float[] delta = Plugin.Instance.GetLastAirMousePositionChange();
    if (delta != null)
    {
        print($"AirMouse position change {delta[0]}, {delta[1]}");
    }
}

void Update()
{
    UpdateAirMousePositionChange();
    //...
}

Orientation

Functionality for exposing IMU based quaternion values. These values represent the orientation of your hand in a quaternion representation.

Mudra.OnImuQuaternionReady onImuQuaternionReady = new Mudra.OnImuQuaternionReady() {
    @Override
    public void run(float[] data) {
        // Do something with data (quaternion, 4 values) 
    }
};

mudra.setOnImuQuaternionReady(onImuQuaternionReady); // Enable 
mudra.setOnImuQuaternionReady(null); // Disable
void OnImuQuaternionPackageReady(vector<float>& data)
{
    cout << "\nReceived Imu Quaternion data";
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetImuQuaternionPackageReady(OnImuQuaternionPackageReady);
}
static void OnImuQuaternionPackageReady(float[] data)
{
    for(int i=0; i<data.Length; i++)
    { 
        Console.Write("\OnImuQuaternionPackageReady = {0}", data[i]);
    }
}

static void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{   
    device.SetImuQuaternionPackageReady(OnImuQuaternionPackageReady);
}
private void UpdateImuQuaternion()
{
    float[] data = Plugin.Instance.GetLastImuQuaternionPackage();
    if (data == null)
    {
        return;
    }
    string s1 = "";
    foreach (var d in data)
    {
        s1 = s1 + d;
    }
    print($"Imu Quaternion {s1}");
}

void Update()
{
    UpdateImuQuaternion();
    //...
}
device.setOnImuQuaternionPackageReady { (timestamp, values) in print("OnImuQuaternionPackageReady")}

Raw Data

Functionality for exposing raw SNC (Surface Nerve Conductance) sensor values. This function may incur an additional fee (We will send a license with instructions for those who are interested in this functionality).

// Please contact support@getmudra.com for instructions
// Please contact support@getmudra.com for instructions
// Please contact support@getmudra.com for instructions
// Please contact support@getmudra.com for instructions

Device Status

Functionality for exposing if the Mudra device is connected.

Mudra.OnDeviceStatusChanged onDeviceStatusChanged = new Mudra.OnDeviceStatusChanged() {
        @Override
        public void run(boolean status) {
            if(status)
            {
                runOnUiThread(new Thread(new Runnable() {
                    public void run() {
                        TextView device_name= findViewById(R.id.txtDevicesNumber);
                        device_name.setText(mMudra.getBluetoothDevice().getAddress());
                    }
                }));
            }
        }
};

mudra.setOnDeviceStatusChanged(onDeviceStatusChanged); // Enable 
mudra.setOnDeviceStatusChanged(null); // Disable

Battery Level

Functionality for exposing battery status.

Mudra.OnBatteryLevelChanged onBatteryLevelChanged = new Mudra.OnBatteryLevelChanged() {
        @Override
        public void run(int i) {
            runOnUiThread(new Thread(new Runnable() {
                public void run() {
                    TextView txtBattery=findViewById(R.id.txtBatLevel);
                    txtBattery.setText(mMudra.getBatteryLevel() + "%");
                }
            }));
        }
};

Mudra.setOnBatteryLevelChanged(onBatteryLevelChanged); // Enable 
Mudra.setOnBatteryLevelChanged(null); // Disable 

Troubleshooting

Problem OS Solution
Mudra does not connect to host device Android Check your Bluetooth version, we recommend 4.2 and above.
Mudra does not connect to host device All Check if your device's LED flashes red, if so recharge (LED will flash blue)
Mudra gestures are not correctly recognize All Please try again our on-boarding app. Try to follow the instruction and perform another calibration.
Mudra pressure values fluctuate All Please make sure your device is in contact with your skin (no need to be tight)
Multiple scan attemts with no connections established Windows Please remove and re-insert dongle into usb port
No dongle found Windows Make sure you are using the correct dongle and it is properly inserted in the USB port
Missing dll error Windows\Unity Make sure installation created an environment variable path and unpacked the dlls inside. If not, try running Setup.msi as administrator
Could not find com.github.wearable-devices:MudraAndroidSDK Windows\Unity Downgrade Unity to version 2019.2.X

Please contact support@getmudra.com for any additional questions or suggestions.