NAV Navbar
java cpp

Getting Started

Welcome

Welcome to the Mudra SDK! Use this documentation to create your own unique experiences based on the Mudra Inspire ™ device and Mudra API. This page is where you can get all the information you need to develop your unique experience.

Our API includes language bindings for Java (Android), C# (Unity) and c++ (Windows) with Swift for iOS and C# for Windows coming soon. In order to use Mudra in your application, all you need is to register a callback. First, follow the methods below and use auto-connect to instantiate Mudra. Use the methods that begin with set to register the callback you need. Afterwards, create the callback function in your code as detailed below.

Device Modes

Mode Description LED Indication
Pairing Flash green continuously alt text
Connected Flash green for 1 second alt text
Low Battery Quick burst flashing red alt text
Charging Flash blue continuously alt text

Android\iOS

Bluetooth should be turned on, if it is off then the Mudra Inspire application will prompt the user to turn it on.

Windows

Insert a dedicated Bluetooth Receiver into the USB slot. We recommend using the following model: nRF52840 Dongle. Run Setup.msi file for installation on your PC, see instructions below.

API functions

Access the API

Android

// gradle.properties
authToken='<Your Jitpack Token>'
// project build.gradle
allprojects {
    repositories {

        google()
        jcenter()
        maven {
                url 'https://jitpack.io'
                credentials { username authToken }
        }
    }
}
// Add the following implementation to your app **build.gradle** file:
implementation 'com.github.wearable-devices:MudraAndroidSDK:+'

Windows

// Header files
#include "Environment.h"
#include "MudraDevice.h"

Initialization and Connection

Important: Please insert your authentication token in your app gradle.properties file in order to use our sdk. See our github example (see here).

Assuming the device is paired manually (for example - via the Android Bluetooth settings), there is no need for initial connection code. Initialization is exposed via the auto connect function.

// Assuming the device is paired manually
import MudraAndroidSDK.Mudra;
Mudra mudra = Mudra.autoConnectPaired(getContext());
Environment mudra;

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    // Insert additional callbacks for additional functionality
    // IMPORTANT - Always add the connect function below to your initialization
    mudra.Connect(device);
}


int main()
{
    mudra.Scan(OnDeviceDiscovered);
    mudra.SetCalibration("<YOUR_PROJECT_PATH>\\Calibration.txt");
    auto devices = mudra.GetDevices();

    // Do something here (set infinite loop here if you woould like to only listen to callbacks)
}

Fingertip Pressure

Functionality for estimating the amount of fingertip pressure. Returned values range between 0 and 1 and callbacks arrive at a rate of approximately 60fps.

mudra.setOnFingertipPressureReady(onProportionalReady); // Enable 
mudra.setOnFingertipPressureReady(null); // Disable

Mudra.OnProportionalReady onFingertipPressureReady = new Mudra.onFingertipPressureReady() {
    @Override
    public void run(float v) {
        // Do something with v (this value is between 0 and 1) 
    }
};
void OnProportionalReady(float proportional)
{
    cout << "\nProportionalfound = " << proportional;
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnProportionalReady(OnProportionalReady);
}

Gesture Recognition

Functionality for predicting three gesture types: thumb, index and tap. Please check out the Mudra app for more info and example usage.

mudra.setOnGestureReady(OnGestureReady); // Enable 
mudra.setOnGestureReady(null); // Disable

Mudra.OnGestureReady onGestureReady = new Mudra.OnGestureReady() {
    @Override
    public void run(Mudra.GestureType gestureType) {
        switch (gestureType) {
        case Tap:
            // Do something here
            break;
        case Index:
            // Do something here
            break;
        case Thumb:
            // Do something here
            break;
        }
    }
};
void OnGestureReady(GestureType gestureType)
{
    cout << "\nGesturefound = " << Environment::ToString(gestureType);
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetOnGestureReady(OnGestureReady);
}

Air Mouse

Functionality for exposing an air mouse based on IMU quaternions. The Mudra air mouse projects raw quaternion values onto a 2d plane, which can be used to move a cursor.

int mScreenWidth, mScreenHeight; // Size of screen in pixels
mudra.setOnAirMousePositionChanged(OnAirMousePositionChanged); // Enable 
mudra.setOnAirMousePositionChanged(null); // Disable

Mudra.OnAirMousePositionChanged OnAirMousePositionChanged = new Mudra.OnAirMousePositionChanged() {
        @Override
        public void run(float[] floats) {

            float airMousePosX += floats[0] * mScreenWidth * HSPEED;
            float airMousePosY += floats[1] * mScreenHeight * VSPEED;
            float airMousePosX = Clamp(mAirMousePosX, 0, mScreenWidth);
            float airMousePosY = Clamp(mAirMousePosY, 0, mScreenHeight);
        }
    };

Orientation

Functionality for exposing IMU based quaternion values. These values represent the orientation of your hand in a quaternion representation.

mudra.setOnImuQuaternionReady(OnImuQuaternionReady); // Enable 
mudra.setOnImuQuaternionReady(null); // Disable

Mudra.OnImuQuaternionReady onImuQuaternionReady = new Mudra.OnImuQuaternionReady() {
    @Override
    public void run(float[] data) {
        // Do something with data (quaternion, 4 values) 
    }
};
void OnImuQuaternionPackageReady(vector<float>& data)
{
    cout << "\nReceived Imu Quaternion data";
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetImuQuaternionPackageReady(OnImuAccNormPackageReady);
}

Raw Data

Functionality for exposing raw SNC (Surface Nerve Conductance) sensor values. This function may incur an additional fee.

mudra.setLicense(Feature.RawData, LICENSE); // private license 
mudra.setOnSncReady(OnSncReady); // Enable 
mudra.setOnSncReady(null); // Disable

Mudra.OnSncReady onSncReady = new Mudra.OnSncReady() {
    @Override
    public void run(float[] floats) {
        // Packet data (3 sensors with 18 samples each - [0...17, 18...35, 36...53])
    }
};
void OnImuAccNormPackageReady(vector<float>& data)
{
    cout << "\nReceived Imu Norm data";
}

void OnSncPackageReady(const SncPackageData& data)
{
    cout << "\nReceived SNC data";
}

void OnDeviceDiscovered(std::shared_ptr<MudraDevice> device)
{
    device->SetImuAccNormPackageReady(OnImuQuaternionPackageReady);
    device->SetOnSncPackageReady(OnSncPackageReady);
}

Device Status

Functionality for exposing if the Mudra device is connected.

mudra.setOnDeviceStatusChanged(onDeviceStatusChanged); // Enable 
mudra.setOnDeviceStatusChanged(null); // Disable

Mudra.OnDeviceStatusChanged onDeviceStatusChanged = new Mudra.OnDeviceStatusChanged() {
        @Override
        public void run(boolean status) {
            if(status)
            {
                runOnUiThread(new Thread(new Runnable() {
                    public void run() {
                        TextView device_name= findViewById(R.id.txtDevicesNumber);
                        device_name.setText(mMudra.getBluetoothDevice().getAddress());
                    }
                }));
            }
        }
};

Battery Level

Functionality for exposing battery status.

Mudra.setOnBatteryLevelChanged(onBatteryLevelChanged); // Enable 
Mudra.setOnBatteryLevelChanged(null); // Disable 

Mudra.OnBatteryLevelChanged onBatteryLevelChanged = new Mudra.OnBatteryLevelChanged() {
        @Override
        public void run(int i) {
            runOnUiThread(new Thread(new Runnable() {
                public void run() {
                    TextView txtBattery=findViewById(R.id.txtBatLevel);
                    txtBattery.setText(mMudra.getBatteryLevel() + "%");
                }
            }));
        }
    };

Troubleshooting

Problem Solution
Mudra does not connect to Android host Check your Bluetooth version, we recommend 4.2 and above.
Check if your device flashes red, if so recharge (LED will flash blue)
Mudra gestures are not correctly recognize Please try again our on-boarding app. Try to follow the instruction and perform another calibration.
Mudra pressure values fluctuate Please make sure your device is in contact with your skin (no need to be tight)
Windows - Multiple scan attemts with no connections established Please remove and re-insert dongle into usb port
Windows - No dongle found Make sure you are using the correct dongle and it is properly inserted in the USB port

Please contact support@wearabledevices.co.il for any additional questions or suggestions.