NAV Navbar
Java c++ c#

Getting Started

Welcome

Welcome to the Mudra SDK! Use this documentation to create your own unique experiences based on the Mudra Inspire ™ device and Mudra API. This page is where you can get all the information you need to develop your unique experience.

Our API includes language bindings for Java (Android), C# (Windows, Unity) and c++ (Windows) with Swift for iOS coming soon. In order to use Mudra in your application, all you need is to register a callback. First, follow the methods below and use auto-connect to instantiate Mudra. Use the methods that begin with set to register the callback you need. Afterwards, create the callback function in your code as detailed below.

Device Modes

Mode Description LED Indication
Pairing Flash green continuously alt text
Connected Flash green for 1 second alt text
Low Battery Quick burst flashing red alt text
Charging Flash blue continuously alt text

Android\iOS

Bluetooth should be turned on, if it is off then the Mudra Inspire application will prompt the user to turn it on.

Windows

Insert a dedicated Bluetooth Receiver into the USB slot. Please use the following model: nRF52840 Dongle. See instructions below for installation and usage. Please install nRF connect , which will automatically install usb ble drivers.

API functions

Access the API

Android

// gradle.properties
authToken='<Your Jitpack Token>'
// Project build.gradle
allprojects {
    repositories {

        google()
        jcenter()
        maven {
                url 'https://jitpack.io'
                credentials { username authToken }
        }
    }
}

// Java 8 support
compileOptions {
    sourceCompatibility 1.8
    targetCompatibility 1.8
}
// App build.gradle file:
implementation 'com.github.wearable-devices:MudraAndroidSDK:+'

// Bluetooth support
implementation 'no.nordicsemi.android:ble:2.1.1'
implementation 'no.nordicsemi.android.support.v18:scanner:1.4.0'

// Android manifest file:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

Windows

// Header files
#include "MudraEnvironment.h"
#include "MudraDevice.h"

Initialization and Connection

Android

Important: Please insert your authentication token in your app gradle.properties file in order to use our sdk. See our github example (see here).

Assuming the device is paired manually (for example - via the Android Bluetooth settings), there is no need for initial connection code. Initialization is exposed via the auto connect function.

Windows

Please call the Scan() function (note the indication in the console window) and press the button on you Mudra device. Once the a Mudra device is discovered, the OnDeviceDiscovered() callback will be called. It is possible to check the name of the new discovered device and filter it if it is not you device. Please note that you need to declare on which hand the device is worn, see SetHand().
When you would like to terminate the connection, please call Close(). Please note that for c++ sdk usage, _WINDOWS define should be included in the c\c++ preprocessor properties.

Logging

It is possible to set the logging severity by the following levels: Debug, Info, Warning and Error. You can set the logging severity using the SetLoggingSeverity(). It is also possible to use your own function for logging using the SetOnLoggingMessageCallback() function.

// Assuming the device is paired manually.
import MudraAndroidSDK.Mudra;
Mudra mudra = Mudra.autoConnectPaired(getContext());
MudraEnvironment mudra;

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{   
    if (device->GetName() != "MY_DEVICE_NAME")
                return;

    // Insert all your device's callbacks here. 

    // You can connect your device on discovery like presented here, or save your device and connect it later. 
    device->SetHand(HandType::Left);
    mudra.Connect(device);
}

void OnLoggingMessageCallback(const string &msg)
{
    // Do something with msg
}

int main()
{
    // Logging
    mudra.SetLoggingSeverity(Logger::Severity::Debug);
    mudra.SetOnLoggingMessageCallback(OnLoggingMessageCallback);

    // Connection
    mudra.Scan(OnDeviceDiscovered);
    mudra.SetCalibration("<YOUR_PROJECT_PATH>\\Calibration.txt");

    // Do something here (set infinite loop here if you woould like to only listen to callbacks).

    // Close connection
    mudra.Close();
}
using Mudra.SDK.Windows.DotNetFramework;

static void OnMudraDeviceDiscovered(MudraDevice device)
{   
    if (device.Name() != "MY_DEVICE_NAME")
                return;

    // Insert all your device's callbacks here. 

    // You can connect your device on discovery like presented here, or save your device and connect it later. 
    device.SetHand(HandType.Left);
    MudraEnvironment.Connect(device);
}

static void OnLoggingMessageCallback(string &msg)
{
    // Do something with msg
}

static void main(string[] args)
{
    // Logging
    MudraEnvironment.SetLoggingSeverity(Logger::Severity::Debug);
    MudraEnvironment.SetOnLoggingMessageCallback(OnLoggingMessageCallback);

    // Connection
    MudraEnvironment.Scan(OnMudraDeviceDiscovered);
    MudraEnvironment.SetCalibration(HandType.Left, "<YOUR_PROJECT_PATH>\\Calibration.txt");

    // Do something here (set infinite loop here if you woould like to only listen to callbacks).

    // Close connection
    MudraEnvironment.Close();
}

Fingertip Pressure

Functionality for estimating the amount of fingertip pressure. Returned values range between 0 and 1 and callbacks arrive at a rate of approximately 60fps.

Mudra.onFingertipPressureReady onFingertipPressureReady = new Mudra.onFingertipPressureReady() {
    @Override
    public void run(float v) {
        // Do something with v (this value is between 0 and 1) 
    }
};

mudra.setOnFingertipPressureReady(onFingertipPressureReady); // Enable 
mudra.setOnFingertipPressureReady(null); // Disable
void OnProportionalReady(float proportional)
{
    cout << "\nProportionalfound = " << proportional;
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnProportionalReady(OnProportionalReady);
}
static void OnProportionalReady(float proportional)
{
    Console.Write("\nProportionalfound = {0}", proportional);
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetOnProportionalReady(OnProportionalReady);
}

Gesture Recognition

Functionality for predicting three gesture types: thumb, index and tap. Please check out the Mudra app for more info and example usage.

Mudra.OnGestureReady onGestureReady = new Mudra.OnGestureReady() {
    @Override
    public void run(Mudra.GestureType gestureType) {
        switch (gestureType) {
        case Tap:
            // Do something here
            break;
        case Index:
            // Do something here
            break;
        case Thumb:
            // Do something here
            break;
        }
    }
};

mudra.setOnGestureReady(onGestureReady); // Enable 
mudra.setOnGestureReady(null); // Disable
void OnGestureReady(GestureType gestureType)
{
    cout << "\nGesturefound = " << MudraEnvironment::ToString(gestureType);
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnGestureReady(OnGestureReady);
}
static void OnGestureReady(GestureType gesture)
{
    Console.Write("\OnGestureReady = {0}", gesture);
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetOnGestureReady(OnGestureReady);
}

Air Mouse

Functionality for exposing an air mouse based on IMU quaternions. The Mudra air mouse projects raw quaternion values onto a 2d plane, which can be used to move a cursor.

int mScreenWidth, mScreenHeight; // Size of screen in pixels

Mudra.OnAirMousePositionChanged onAirMousePositionChanged = new Mudra.OnAirMousePositionChanged() {
        @Override
        public void run(float[] floats) {

            float airMousePosX += floats[0] * mScreenWidth * HSPEED;
            float airMousePosY += floats[1] * mScreenHeight * VSPEED;
            float airMousePosX = Clamp(mAirMousePosX, 0, mScreenWidth);
            float airMousePosY = Clamp(mAirMousePosY, 0, mScreenHeight);
        }
};

mudra.setOnAirMousePositionChanged(onAirMousePositionChanged); // Enable 
mudra.setOnAirMousePositionChanged(null); // Disable
void OnAirMousePositionChanged(const vector<float>& position)
{
    cout << "\nX=" << position[0] << " ,Y=" << position[1];
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnAirMousePositionChanged(OnAirMousePositionChanged);
}
static void OnAirMousePositionChanged(float x, float y)
{
    Console.Write("\OnAirMousePositionChanged = {0}, {1}", x, y);
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetOnAirMousePositionChanged(OnAirMousePositionChanged);
}

Orientation

Functionality for exposing IMU based quaternion values. These values represent the orientation of your hand in a quaternion representation.

Mudra.OnImuQuaternionReady onImuQuaternionReady = new Mudra.OnImuQuaternionReady() {
    @Override
    public void run(float[] data) {
        // Do something with data (quaternion, 4 values) 
    }
};

mudra.setOnImuQuaternionReady(onImuQuaternionReady); // Enable 
mudra.setOnImuQuaternionReady(null); // Disable
void OnImuQuaternionPackageReady(vector<float>& data)
{
    cout << "\nReceived Imu Quaternion data";
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetImuQuaternionPackageReady(OnImuAccNormPackageReady);
}
static void OnImuQuaternionPackageReady(float[] data)
{
    for(int i=0; i<data.Length; i++)
    { 
        Console.Write("\OnImuQuaternionPackageReady = {0}", data[i]);
    }
}

static void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{   
    device.SetImuQuaternionPackageReady(OnImuAccNormPackageReady);
}

Raw Data

Functionality for exposing raw SNC (Surface Nerve Conductance) sensor values. This function may incur an additional fee.

Mudra.OnSncReady onSncReady = new Mudra.OnSncReady() {
    @Override
    public void run(float[] floats) {
        // Packet data (3 sensors with 18 samples each - [0...17, 18...35, 36...53])
    }
};

mudra.setLicense(Feature.RawData, LICENSE); // private license 
mudra.setOnSncReady(onSncReady); // Enable 
mudra.setOnSncReady(null); // Disable
void OnImuAccNormPackageReady(vector<float>& data)
{
    cout << "\nReceived Imu Norm data";
}

void OnSncPackageReady(const SncPackageData& data)
{
    cout << "\nReceived SNC data";
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetImuAccNormPackageReady(OnImuQuaternionPackageReady);
    device->SetOnSncPackageReady(OnSncPackageReady);
}
static void OnImuAccNormPackageReady(float[] data)
{
    for(int i=0; i<data.Length; i++)
    { 
        Console.Write("\OnImuAccNormPackageReady = {0}", data[i]);
    }
}

static void OnSncPackageReady(int snc, float[] data)
{
    Console.Write("\OnSncPackageReady snc = {0}", snc);

    for(int i=0; i<data.Length; i++)
    { 
        Console.Write("\OnSncPackageReady = {0}", data[i]);
    }
}

static void OnDeviceDiscovered(MudraDevice device)
{
    device.SetImuAccNormPackageReady(OnImuQuaternionPackageReady);
    device.SetOnSncPackageReady(OnSncPackageReady);
}

Device Status

Functionality for exposing if the Mudra device is connected.

Mudra.OnDeviceStatusChanged onDeviceStatusChanged = new Mudra.OnDeviceStatusChanged() {
        @Override
        public void run(boolean status) {
            if(status)
            {
                runOnUiThread(new Thread(new Runnable() {
                    public void run() {
                        TextView device_name= findViewById(R.id.txtDevicesNumber);
                        device_name.setText(mMudra.getBluetoothDevice().getAddress());
                    }
                }));
            }
        }
};

mudra.setOnDeviceStatusChanged(onDeviceStatusChanged); // Enable 
mudra.setOnDeviceStatusChanged(null); // Disable

Battery Level

Functionality for exposing battery status.

Mudra.OnBatteryLevelChanged onBatteryLevelChanged = new Mudra.OnBatteryLevelChanged() {
        @Override
        public void run(int i) {
            runOnUiThread(new Thread(new Runnable() {
                public void run() {
                    TextView txtBattery=findViewById(R.id.txtBatLevel);
                    txtBattery.setText(mMudra.getBatteryLevel() + "%");
                }
            }));
        }
};

Mudra.setOnBatteryLevelChanged(onBatteryLevelChanged); // Enable 
Mudra.setOnBatteryLevelChanged(null); // Disable 

Troubleshooting

Problem OS Solution
Mudra does not connect to host device Android Check your Bluetooth version, we recommend 4.2 and above.
Mudra does not connect to host device All Check if your device's LED flashes red, if so recharge (LED will flash blue)
Mudra gestures are not correctly recognize All Please try again our on-boarding app. Try to follow the instruction and perform another calibration.
Mudra pressure values fluctuate All Please make sure your device is in contact with your skin (no need to be tight)
Multiple scan attemts with no connections established Windows Please remove and re-insert dongle into usb port
No dongle found Windows Make sure you are using the correct dongle and it is properly inserted in the USB port

Please contact support@wearabledevices.co.il for any additional questions or suggestions.