NeuroIK Documentation

Desktop robot design, analysis, training & deployment

1Overview

NeuroIK is a desktop application for designing a robot arm, understanding how it behaves, and getting it to do something useful. It covers the whole path from a folder of CAD meshes to a trained model on an ESP32 — and every stage in between is a workflow you can use on its own.

1.1 The six workflows

The landing page offers six. They share one robot description, so a robot built or imported in one is immediately available to the rest — with its joint limits, link lengths and mass properties carried through rather than re-entered.

Which one do I want? If you have meshes and no robot, start at CAD → URDF. If you have a robot and a question about a single pose, use the IK Calculator. If the question is about motion — will it move well, will the motors cope, what path should it take — the Jacobian, Torque & Dynamics and Motion & Path tabs answer those in that order. Reach for ML for IK when you need IK solved fast on hardware that cannot run a solver.

1.2 What is in the box

Robot types

SCARA, Delta, Cartesian, Articulated (2–12 DOF), or import your own URDF/Xacro

Architectures

MLP, Transformer, GNN, Diffusion, PINN, RL Agent

Export targets

ONNX, PyTorch, TorchScript, TF Lite, Keras H5, TensorRT, ROS2, TinyML, native DLL, Core ML

Under the hood

HTML5 / CSS3 / Vanilla JS + Three.js + Chart.js, on a Python 3.11 · FastAPI · PyTorch 2 engine

1.3 This workflow: teaching a network to solve IK

The rest of this tab is the ML for IK workflow: design a robot, generate a dataset from it, train a model, check the model in a live 3D viewport, and export it for deployment — from an NVIDIA GPU down to an ESP32.

The core idea: FK-first data

Inverse kinematics asks: “which joint angles put the gripper here?” Generating training data by guessing poses would fill the dataset with unreachable targets. NeuroIK inverts the problem:

  1. Your recipe defines each joint's limits.
  2. Random joint values are sampled within those limits.
  3. Forward kinematics computes exactly where the end-effector lands.

Every row is therefore reachable by construction, and the network learns the inverse of a guaranteed-correct mapping. No impossible poses ever enter training.

Note: IK is one-to-many — many joint configurations reach the same pose. NeuroIK trains against a forward-kinematics (task-space) loss, rewarding any joint vector that reaches the target, rather than forcing the network to average conflicting valid answers.
NeuroIK projects screen
The landing screen — create a new project or reopen a previous one with its recipe, datasets and trained models restored.

2Installation & Launch

2.1 System requirements

ComponentMinimumRecommended
OSWindows 10/11, macOS 12+, Ubuntu 22.04+Windows 11
Python3.11+3.11
RAM8 GB16 GB
GPUNone — CPU fallbackNVIDIA, CUDA 11.8+
Disk2 GB8 GB (PyTorch + TensorFlow)

2.2 Install & run

Put the NeuroIK folder anywhere you like and double-click NeuroIK.bat. That is the whole installation.

On the first run it installs everything NeuroIK needs and then opens the app. Every run after that skips straight to opening the app.

Heads-up — first run only: a console window appears while the dependencies download. This is a few GB and takes several minutes; leave it alone until it finishes and the app window appears. Later launches are silent and much faster.
Tip: want an icon on your Desktop? Run python desktop.py --install-shortcut once. Afterwards you can start NeuroIK from the Desktop icon instead of the batch file — no console, just the app.

Closing the app window shuts everything down; nothing keeps running in the background.

2.3 Where files are stored

Everything the app produces lives under a single data/ folder:

NeuroIK/
└── data/
    ├── datasets/   # generated CSVs
    ├── models/     # training checkpoints (best_model.pt)
    ├── exports/    # exported models, one subfolder per format
    └── projects/   # project state (JSON)

You can additionally mirror datasets and exports to a folder of your choosing — see Project Management.

2.4 A portable build

NeuroIK can be packaged into a folder that runs on any Windows machine with no Python installed — copy it to a USB stick and go.

python build_exe.py --venv buildenv\Scripts\python.exe --zip

The result is dist/NeuroIK/; run NeuroIK.exe inside it. Your projects are written to a data/ folder created beside the executable, so the whole folder stays self-contained and moves with your work.

One real trade. The portable build uses the CPU-only PyTorch wheel, because the CUDA one is around 4.5 GB against 200 MB — and needs a matching NVIDIA driver on the target machine anyway. GPU training is therefore unavailable in the portable copy; use the source install for that. TensorFlow and coremltools are also left out, which costs the TF Lite, Keras H5 and Core ML export formats (3 of 10). Everything else — all six workflows, CPU training of every architecture, ONNX and the remaining exports — works.

Full details, including what to do if it will not start, are in docs/PORTABLE-BUILD.md.

2.5 First run

NeuroIK opens on the Projects screen with no projects yet. Name a project, click Create Project, and you land on Robot Selection — step 1 of the workflow numbered in the sidebar.

3Robot Selection & Configuration

Robot selection screen
Step 1 — pick the kinematic family. Each card shows the DOF range and typical uses.

3.1 Available robot types

TypeDOFDescriptionBest for
SCARA3–4Selective Compliance Assembly Robot Arm — fast planar motion with vertical reachPick-and-place, assembly, PCB manufacturing
Delta3 (fixed)Parallel linkage — three arms 120° apartPackaging, food, pharma, high-speed sorting
Cartesian3–6Linear X/Y/Z travel, optional roll/pitch/yaw wristCNC, 3D printing, lab automation, gantries
Articulated2–12Serial manipulator with rotary joints — the most versatileWelding, painting, material handling
Custom URDF2–12Import your own robot from a .urdf/.xacro file — exact chain + your CAD meshesReal hardware you already have a description for

SCARA

Two revolute joints sweep in a horizontal plane, a prismatic Z axis mounted at the elbow provides vertical travel, and an optional 4th revolute wrist spins the tool. IK: closed-form analytical. Parameters: link lengths L1/L2 (m), Z travel range, optional wrist limits.

Delta

Three arms mounted 120° apart drive a floating platform; the robot hangs from its base and reaches below it, so its Z workspace is negative. IK: trilateration. Parameters: exactly four — [R_base, R_platform, L_upper, L_lower].

Note: a Delta is a 3-DOF translation-only machine — orientation is always zero. Those constant pose dimensions are detected and masked during training.

Cartesian

Three prismatic axes map 1:1 onto X/Y/Z, with an optional wrist. IK: direct mapping — no iteration needed. Parameters: travel range per axis.

Articulated

An anthropomorphic chain: a base yaw joint and column, parallel-pitch shoulder and elbow, and — at 6+ DOF — a spherical wrist (roll · pitch · roll) whose axes meet at a single point. DH parameters are generated automatically from your link lengths, and the home pose stands straight up along +Z. IK: numerical damped least-squares with 12 multi-start restarts.

Custom URDF

Instead of describing a robot by hand, import one. Click Custom URDF and enter a path — either the .urdf/.xacro file, or the folder containing it (recommended: mesh references resolve relative to that folder).

NeuroIK walks the real chain and uses each joint's true origin (xyz + rpy) and axis, so imported robots run on a dedicated exact-kinematics engine — not the auto-generated DH approximation used by the generic Articulated type. That exactness is what lets your CAD line up with the joint motion. IK: numerical damped least-squares with multi-start restarts.

What gets imported
  • Joint count, types and limits — revolute limits convert from radians to degrees; prismatic stays in metres. continuous joints and joints with no <limit> default to ±180° (you'll get a warning).
  • Link lengths — the distance from each actuated joint to the next, with any intervening fixed joints folded in.
  • Visual geometry — rendered in the Live Preview and Simulation.
Mesh formats

STL, DAE and OBJ are rendered, as are URDF <box>/<cylinder>/<sphere> primitives. Paths are resolved through package://, file://, absolute and relative forms — and finally by filename anywhere under the folder, which covers descriptions whose package:// paths don't match the shipped layout.

STEP / IGES are not rendered. They're B-rep CAD formats that need a CAD kernel to tessellate into triangles — a browser can't. Those links show a translucent placeholder; kinematics, datasets and training are unaffected.
Xacro support

.xacro files are expanded internally — no ROS install required: <xacro:property>, ${...} arithmetic, <xacro:include> and <xacro:if>/<xacro:unless>. Expressions are evaluated over a restricted AST, never eval().

<xacro:macro> is not supported. Expanding macros correctly is a large job, and a partial implementation would silently produce a wrong robot. Pre-expand instead: xacro robot.xacro -o robot.urdf, then import the generated robot.urdf.
Branches and mimic joints

The whole tree is drawn, not just the actuated chain — so side branches like a gripper's second finger appear. The DOF, however, come from the longest actuated chain.

A <mimic> joint is driven by another joint, so it is not an independent DOF: it's excluded from the joint count (and from the network's outputs) but follows its source in 3D via value = multiplier × source + offset.

Editing after import

Joint limits and link lengths stay editable — narrowing limits is a legitimate way to shape the dataset, and changing a link length rescales the spacing to the next joint (kinematics and the 3D view stay in step). The DOF count and joint types are locked by the file: the engine rejects a joint count that disagrees with the parsed chain, and flipping a type would corrupt the degrees/metres conversion.

Note: CAD meshes are rigid parts — they keep their own size when you change a link length, so a large change will show a visible gap between links.
Only some of my meshes appear

NeuroIK draws the links the URDF declares. If a folder ships 13 STLs but the description references 7, you'll see 7 — the rest are spare parts no link points at. Add <link>/<joint> entries to use them.

3.2 Recipe Builder

Recipe builder
Step 2 — link lengths, joint types and limits, dataset size, plus a live 3D preview of the arm you are describing.

The Recipe Builder is where a robot becomes concrete:

  • Link lengths — each joint's link, in metres.
  • Joint type — revolute (degrees) or prismatic (metres).
  • Joint limits — min/max per joint. These bound the sampling, so they define the reachable workspace.
  • Dataset size — 10K, 100K or 1M samples.
  • Name — saved with the project.

Live Robot Preview

The preview renders through the same code as the Simulation tab, so what you see here is what you'll simulate. It updates as you type — no need to regenerate a dataset first.

  • Joint Controls — one slider per joint, bounded to the limits you set, to pose the arm and sanity-check its reach before committing to a dataset.
  • SCARA J3 — the vertical prismatic travel is derived from the 3rd link length (the quill descends from 0 to −L3), so those limit fields are read-only and follow the link.
  • Imported URDFs are framed automatically, since they arrive at any scale.
Saved recipe (JSON)
{
  "robot_type": "articulated",
  "num_joints": 6,
  "link_lengths": [0.2, 0.4, 0.4, 0.1, 0.1, 0.05],
  "joints": [
    { "joint_type": "revolute", "min_limit": -180, "max_limit": 180 }
  ],
  "recipe_name": "My Robot",
  "dataset_size": 100000
}
Warning: the recipe defines the model's input/output shape. If you change link lengths or joint count afterwards, regenerate the dataset and retrain — an old model no longer describes the new robot.

4Dataset Generation

Click Generate Dataset in the Recipe Builder. Generation runs in the background with a live progress bar; the UI stays responsive.

Output format

A single CSV — one row per sample:

x,y,z,roll,pitch,yaw,joint_1,joint_2,...,joint_N
ColumnsMeaningUnits
x, y, zEnd-effector positionmetres
roll, pitch, yawEnd-effector orientationradians
joint_1 … joint_NJoint values that produce that posedegrees (revolute) / metres (prismatic)

Workspace analytics

After generation NeuroIK analyses the reachable volume (x/y/z ranges) and uses it to calibrate the Simulation sliders — so default targets land where the robot can actually go. This is why a Delta's Z slider is negative: it reaches below its base.

Tip: 10K is fine to smoke-test the pipeline. Use 100K+ for a production-quality model. 1M with a complex chain can take several minutes.

5ML Studio

ML Studio
Step 3 — choose an architecture, set hyperparameters, and watch training stream live. Hover any ⓘ for a definition and its target value.

GPU or CPU — chosen automatically

NeuroIK picks the training device at run time: CUDA if a usable GPU is present, CPU otherwise. There is nothing to configure and nothing breaks without a GPU. A banner at the top of ML Studio always states which device the next run will use.

Have an NVIDIA GPU but training is still on the CPU? This catches almost everyone on Windows. PyPI's default torch wheel is CPU-only, so a plain pip install torch gives you a build where torch.cuda.is_available() is permanently False — the GPU is never touched, however capable it is. Check with:

python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

A build ending in +cpu (e.g. 2.2.2+cpu) is the CPU-only wheel. Install the CUDA build instead:

pip install --force-reinstall torch --index-url https://download.pytorch.org/whl/cu121

NeuroIK.bat now does this for you on first install: it looks for nvidia-smi and pulls the CUDA wheel when a GPU is present.

ML Studio detects this exact situation — GPU present, torch unable to use it — by asking the driver via nvidia-smi rather than trusting torch, and shows the fix inline. The /api/system endpoint reports the same thing:

{
  "torch_version": "2.2.2+cpu",
  "torch_cuda_build": null,
  "cuda_available": false,
  "device": "cpu",
  "gpu_name": "NVIDIA GeForce MX550",
  "gpu_present_but_unusable": true,
  "gpu_hint": "... install a CUDA build to use the GPU"
}
SituationDevice usedWhat ML Studio shows
CUDA build + NVIDIA GPUcudaGreen — "Training on GPU", card name and VRAM
No NVIDIA GPUcpuBlue — "Training on CPU", core count
GPU present, CPU-only torchcpuAmber — the card name plus the exact install command
When on GPU NeuroIK enables pinned host memory and non-blocking transfers so the device is not left waiting on data. Both are disabled on CPU, where they would be pure overhead. Live GPU utilisation and memory appear in the metric cards during training.
Set expectations: IK networks are small (a default MLP is ~564K parameters). On an entry-level laptop GPU the speed-up over a modern multi-core CPU is real but modest — the win grows with dataset size and batch size. A 1M-row dataset benefits far more than a 10K one.

Training configuration

SettingRangeDefaultNotes
Epochs1–100050Full passes over the data
Batch size16–4096256Larger = smoother, more memory
Learning rate0.0001–1.00.001Auto-reduced on plateau
OptimizerAdam · AdamW · SGD · RMSPropAdam/AdamW are robust defaults
Train/test split50–95%80%Held-out validation share

Recommended starting point for 100K samples

EpochsBatchLROptimizerSplit
6010240.001Adam0.85

Live metrics

Training streams in real time: train/val loss, accuracy %, RMSE, MAE, learning rate, elapsed, ETA, CPU/GPU usage — plus two charts.

MetricMeaningTarget for a good model
Accuracy %Share of validation targets reached within tolerance (checked by FK)≥ 90% strong, ≥ 95% excellent
RMSERoot-mean-square end-effector position error< 0.02 m
MAETypical miss distance< 0.01 m
Val lossError on held-out dataTracks train loss; a widening gap = overfitting
Note: RMSE and MAE are reported as end-effector position error in metres, not joint error. Because IK has many valid answers, task-space error is the honest measure.

The two charts

  • Loss curves — train vs validation per epoch. Both should fall then flatten.
  • Target vs Predicted — the pose the model actually reaches (FK on its predicted joints) against the requested target, for x/y/z. A perfect model puts every point on the dashed y = x line; scatter away from the diagonal means the arm misses.

5.1 Architectures

MLP — Multi-Layer Perceptron

Structure: 6 → 256 → 512 → 512 → 256 → 128 → N, GELU activations, LayerNorm, dropout.
Data: 10K–100K  ·  Best for: every robot type, as a fast baseline

  • Pros — fastest to train, lowest memory, excellent on SCARA/Cartesian/Delta. The only architecture that exports to a complete TinyML header.
  • Cons — no built-in mechanism for multi-modal solutions; can plateau on long articulated chains.

Transformer

Structure: encoder over 6 learned token embeddings, multi-head self-attention, LayerNorm, learned positional encoding.
Data: 100K+  ·  Best for: articulated / SCARA with long-range coupling

  • Pros — attention captures dependencies between pose dimensions and joints.
  • Cons — more memory, slower, hungrier for data.

GNN — Graph Neural Network

Structure: message passing over the kinematic chain — joints are nodes, links are edges; 3 rounds of learned node/edge updates.
Data: 50K+  ·  Best for: articulated arms

  • Pros — the network encodes chain topology explicitly.
  • Cons — overkill when the structure is trivial (Cartesian).

Diffusion — DDPM

Structure: denoises random joint vectors conditioned on the target pose; cosine schedule, FiLM conditioning. At inference it samples 16 candidates and keeps whichever FK says lands closest.
Data: 100K+  ·  Best for: poses with several valid solutions

  • Pros — handles multi-modal IK naturally; strongest in ambiguous cases.
  • Cons — ~16× slower inference, heaviest to train.

PINN — Physics-Informed

Structure: MLP backbone plus an FK-consistency loss penalising predictions whose FK misses the target.
Data: 10K–50K  ·  Best for: small datasets

  • Pros — best sample efficiency; physically consistent.
  • Cons — dual objective can be unstable if the FK weight is off.

RL Agent

Structure: actor-critic (PPO-style) with behaviour-cloning pretraining; reward is pose error.
Data: 100K+  ·  Best for: research and experimentation

  • Pros — can discover strategies absent from the data.
  • Cons — needs reward tuning; non-deterministic outputs.

5.2 Choosing a model

New to NeuroIK?
  └─ Start with MLP — fast baseline, works for everything
     └─ Accuracy acceptable? ──► Done. Use MLP.
        └─ No ─► Robot has 6+ joints?
           ├─ Yes ─► Transformer or GNN
           └─ No  ─► Diffusion (multi-modal) or PINN (small data)
                     └─ Still not enough? ─► RL Agent
Tip: deploying to a microcontroller? Choose MLP from the start — it is the only architecture that compiles to a complete standalone C header.

6Simulation Studio

Simulation Studio 3D viewport
Step 4 — a real-scale parametric 3D model built from your recipe. Here: a 6-DOF articulated arm in its straight-up home pose.

6.1 The 3D viewport

  • Controls — drag to orbit, scroll to zoom, right-drag to pan.
  • Real scale — everything is drawn in metres, matching the recipe.
  • Markers — a green sphere marks the target; a blue sphere marks where the model actually put the end-effector.
  • Collisions — for articulated arms, non-adjacent links are AABB-checked; colliding pairs turn red.
Note on axes: the viewport is Z-up — Z is vertical, matching the engine's convention. The Z slider raises the target; Y moves it in depth.

6.2 Running inference

Set the target with six controls — X/Y/Z in metres, roll/pitch/yaw in degrees — then press Predict & Animate. Each slider has a number box beside it for exact entry; the box is authoritative, so it also accepts values outside the slider's range when you need a target the slider can't reach.

Prediction results panel
Prediction results: ML joints vs analytical IK, error metrics, and a target-vs-predicted breakdown per axis.

The results panel reports:

FieldMeaning
Predicted joint valuesRaw output of the trained network
Analytical joint valuesClassical solver's answer, for comparison
IK Match badgeGreen when position error < 2 cm and orientation error < 5°
Pos / Ori errorHow far the end-effector landed from the target (m / °)
Inference timeModel latency in ms
Note: the ML and analytical joint values often differ while both reach the target — that is IK's one-to-many nature, not an error. Trust the position error, not the joint diff.

The ⓘ icon next to Analytical Joint Values opens the actual IK formulas for the loaded robot — the closed-form SCARA solution, the Delta per-arm trilateration, the Cartesian direct mapping, or the damped-least-squares iteration used for Articulated and imported URDF robots.

"Unreachable / No IK" on a target that looks reachable? The badge reports that the analytical solver found no solution. Check the target is genuinely within reach — a SCARA with L1+L2 = 0.7 m cannot reach 0.82 m, and no solver (neural or classical) can change that. The ML model will still return its closest attempt at the workspace edge.

6.3 Random target

Random Target gives a guaranteed-reachable pose. If the project has a generated dataset it returns an actual row from it — the exact distribution the model was trained on, which is the fairest way to spot-check accuracy. Otherwise it samples random joints within your limits and runs FK. Either way the pose is reachable by construction, so a blind bounding-box guess never lands outside the workspace.

6.4 Export panel

After a successful prediction, the Export Model panel appears beneath the viewport with all ten targets. Hover any ⓘ for a one-line summary of what a format is for, and see §7 for integration code. Formats that cannot represent the current architecture report a clear message instead of exporting — check the support matrix first.

7Exporting Models

Export formats panel
Ten deployment targets. Hover any ⓘ to see what each format is for.

Every export reports format_name, file_size_mb, param_count, supported_runtime and validation_passed, and is written to data/exports/<format>/.

7.1 Normalization is baked in

NeuroIK trains on normalized values, but you never have to think about that. Every export folds the normalization constants into the model itself: feed a raw pose in metres and radians, read real joint units — degrees for revolute joints, metres for prismatic — straight back out.

joints = model(pose)          // that's the whole contract

Each export still ships a model_meta.json beside it, both as documentation and so tooling can read the I/O layout:

model_meta.json
{
  "model_type": "mlp",
  "input_dim": 6,
  "output_dim": 4,
  "pose_order": ["x","y","z","roll","pitch","yaw"],
  "pose_units": "x/y/z in metres, roll/pitch/yaw in radians",
  "joint_units": "degrees for revolute joints, metres for prismatic",
  "normalization": { "x_mean": [...], "x_std": [...],
                      "y_mean": [...], "y_std": [...] },
  "normalization_baked_in": true,
  "usage": "joints = model(pose) - normalization is built into the graph..."
}
Do not apply the constants yourself. They are published for reference only. Applying (pose - x_mean) / x_std before calling a baked model normalizes the input twice and produces confident-looking nonsense. Always check normalization_baked_in — it is true for every format except one.
The single exception: the ROS2 package. Its ik_node.py applies the constants explicitly in predict(), so its model_meta.json reports "normalization_baked_in": false. The node handles it for you; you only need to care if you load best_model.pt yourself.
How it works. Traced formats (ONNX, TorchScript, TensorRT, Core ML) export a thin wrapper module that scales on the way in and out. TF Lite and Keras H5 instead absorb the constants into the first and last Dense layers — an affine op folded into an adjacent affine layer is mathematically identical and costs nothing at inference. TinyML and the native DLL emit the arithmetic directly into their generated C. Every one is verified against PyTorch numerically at export time, so a mis-folded constant fails the export instead of shipping.

7.2 Which formats work with which architecture

Not every architecture can become every format. Formats that rebuild the network layer-by-layer (TF Lite, Keras H5, native DLL) need a plain feed-forward stack; diffusion, GNN, transformer and RL keep their real logic inside forward(), so those combinations are refused at export time rather than silently producing a model that has the right output shape but computes the wrong function.

Architectureonnxpytorchtorchscripttensorrt tfliteh5ros2tinymldllcoreml
mlp OKOK OKOK OKOK OKOK no-ccOK
pinn OKOK OKOK OKOK OKOK no-ccOK
transformer OKOK OKOK refusedrefused OKOK refusedfails
gnn OKOK OKOK refusedrefused OKOK refusedOK
diffusion OKOK OKOK refusedrefused OKOK refusedOK
rl OKOK OKOK refusedrefused OKOK refusedOK

OK exports and passes validation  ·  refused deliberately blocked with an explanatory message  ·  no-cc works, but needs a C compiler installed (the C sources and a build script are written either way)  ·  fails conversion is not possible — see the format's notes.

Reading the matrix: if you plan to deploy to a microcontroller, a native library, TF Lite or Keras, train an MLP (or PINN). If you need a different architecture, ONNX, TorchScript, PyTorch, TensorRT and ROS2 accept all six.
TinyML nuance: the matrix shows OK for every architecture because the export always succeeds — but a runnable ik_forward() is only generated for plain feed-forward models. Others get their raw weight arrays and no forward pass. See 7.10.

7.3 ONNX

File: model.onnx + model_meta.json  ·  Opset: 17, dynamic batch  ·  Runtime: ONNX Runtime, OpenVINO  ·  I/O: pose float32[B,6] → joints float32[B,N]

#include <onnxruntime_cxx_api.h>
#include <vector>

Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "neuroik");
Ort::Session session(env, "model.onnx", Ort::SessionOptions{});

float pose[6]     = {0.5f, 0.3f, 0.2f, 0.0f, 0.0f, 0.0f};
int64_t shape[2]  = {1, 6};

auto mem   = Ort::MemoryInfo::CreateCpu(OrtArenaAllocator, OrtMemTypeDefault);
auto input = Ort::Value::CreateTensor<float>(mem, pose, 6, shape, 2);

const char* in_names[]  = {"pose"};
const char* out_names[] = {"joints"};

auto out = session.Run(Ort::RunOptions{nullptr},
                       in_names, &input, 1, out_names, 1);

float* joints = out[0].GetTensorMutableData<float>();
// joints[0..N-1] are already in real units.

Link against onnxruntime. Ort::Value does not own the buffer, so `pose` must outlive the Run() call.

7.4 PyTorch

File: model.pt  ·  Runtime: PyTorch 2.0+  ·  Use for: continued training, fine-tuning, research

Note: this is a full checkpoint dictionary (weights + architecture metadata + normalization), not a TorchScript archive — so load it with torch.load, not torch.jit.load. That is what makes it re-trainable.
This is the one format where you apply normalization yourself. A raw checkpoint is the bare network — there is no graph to bake anything into. If you just want to run the model rather than train it, use TorchScript instead, which needs none of this boilerplate.
import torch
import numpy as np
from backend.ml.models import create_model

ckpt  = torch.load("model.pt", map_location="cpu", weights_only=False)
model = create_model(ckpt["model_type"], ckpt["input_dim"], ckpt["output_dim"])
model.load_state_dict(ckpt["model_state_dict"])
model.eval()

# This is the ONE format where you normalize yourself -- a bare checkpoint has
# no graph to fold the constants into.
x_mean, x_std = np.array(ckpt["x_mean"]), np.array(ckpt["x_std"])
y_mean, y_std = np.array(ckpt["y_mean"]), np.array(ckpt["y_std"])

pose = np.array([[0.5, 0.3, 0.2, 0.0, 0.0, 0.0]])
x = torch.tensor((pose - x_mean) / x_std, dtype=torch.float32)

with torch.no_grad():
    joints = model(x).numpy() * y_std + y_mean

print(joints[0])

Needs the NeuroIK source on the Python path for create_model.

7.5 TorchScript

File: model_torchscript.pt + model_meta.json  ·  Runtime: LibTorch C++, PyTorch 2.0+, TorchServe  ·  Architectures: all six

A TorchScript module is self-contained: it carries its own architecture, so it loads with no model class, no create_model, and no NeuroIK source on the target machine. The module is traced and then torch.jit.freezed, which inlines the weights and folds away the module hierarchy.

This is the full-precision option for C++ robot controllers — no conversion to a foreign framework, so the numbers match PyTorch to within float round-off (validated at export to 1e-5 of a joint standard deviation).

#include <torch/script.h>
#include <iostream>

int main() {
    torch::jit::script::Module model = torch::jit::load("model_torchscript.pt");
    model.eval();

    // Raw pose -- normalization is traced into the module.
    torch::Tensor pose = torch::tensor({{0.5f, 0.3f, 0.2f, 0.0f, 0.0f, 0.0f}});

    torch::NoGradGuard no_grad;
    torch::Tensor joints = model.forward({pose}).toTensor();

    std::cout << joints << std::endl;   // real joint units
}

Download LibTorch and point CMake at it with -DCMAKE_PREFIX_PATH.

7.6 TF Lite

File: model.tflite + model_meta.json  ·  Runtime: TFLite, Android, iOS, embedded Linux  ·  Architectures: MLP, PINN

Weights are transferred into an equivalent tf.keras functional model — with the normalization constants folded into the first and last Dense layers — then converted and checked against PyTorch through the TFLite interpreter.

import org.tensorflow.lite.Interpreter

val interpreter = Interpreter(loadModelFile("model.tflite"))

// Raw pose: x, y, z (m), roll, pitch, yaw (rad). Normalization is folded
// into the first and last Dense layers.
val pose   = arrayOf(floatArrayOf(0.5f, 0.3f, 0.2f, 0.0f, 0.0f, 0.0f))
val joints = Array(1) { FloatArray(N_JOINTS) }

interpreter.run(pose, joints)
// joints[0] is in degrees / metres

interpreter.close()

implementation "org.tensorflow:tensorflow-lite:2.x" in build.gradle.

Precision note: NeuroIK deliberately does not apply TFLite's default int8 quantization. IK is a precision regression problem — quantization cost about 0.06 of a joint standard deviation in testing, several degrees of real error. These models are only ~2 MB, so the size saving was not worth it.

7.7 Keras H5

File: model.h5 + model_meta.json  ·  Runtime: TensorFlow 2.x / Keras, TF Serving  ·  Architectures: MLP, PINN

The network is rebuilt as a genuine tf.keras Functional model and the weights are transferred layer by layer, so the result is a first-class Keras model — no PyTorch required on the target. Use it to drop IK into an existing TensorFlow pipeline, serve it with TF Serving, or fine-tune it further in Keras.

import numpy as np
import tensorflow as tf

model = tf.keras.models.load_model("model.h5")

# Raw pose -- normalization is inside the first and last layers.
pose = np.array([[0.5, 0.3, 0.2, 0.0, 0.0, 0.0]], dtype=np.float32)

joints = model.predict(pose, verbose=0)
print(joints[0])          # degrees / metres

Needs no PyTorch -- the network was rebuilt as a native Keras model.

7.8 TensorRT

File: model_tensorrt.onnx + model_meta.json  ·  Runtime: TensorRT 8.x+, Triton

An ONNX graph with constant folding applied, ready to be compiled into a TensorRT engine on the target device — engines are hardware-specific and not portable.

#include <NvInfer.h>
#include <fstream>
#include <vector>

// Load the engine you built with trtexec on THIS machine.
std::ifstream f("model.engine", std::ios::binary);
std::vector<char> blob((std::istreambuf_iterator<char>(f)), {});

auto runtime = nvinfer1::createInferRuntime(logger);
auto engine  = runtime->deserializeCudaEngine(blob.data(), blob.size());
auto ctx     = engine->createExecutionContext();

float pose[6] = {0.5f, 0.3f, 0.2f, 0.0f, 0.0f, 0.0f};
float joints[N_JOINTS];

void *d_in, *d_out;
cudaMalloc(&d_in,  sizeof(pose));
cudaMalloc(&d_out, sizeof(joints));
cudaMemcpy(d_in, pose, sizeof(pose), cudaMemcpyHostToDevice);

void* bindings[] = {d_in, d_out};
ctx->executeV2(bindings);

cudaMemcpy(joints, d_out, sizeof(joints), cudaMemcpyDeviceToHost);

Link nvinfer and cudart.

7.9 ROS2 Package

Runtime: ROS2 Humble / Iron  ·  Output: a ready-to-build package  ·  Architectures: all six

The package ships with the weights inside it — no copying files around, no NeuroIK source needed on the robot:

neuroik_ros2/
├── package.xml
├── setup.py                  # package_data ships the weights
├── resource/neuroik_ros2     # ament marker (required for discovery)
└── neuroik_ros2/
    ├── __init__.py
    ├── ik_node.py            # constants + load_model() + predict()
    ├── best_model.pt         # the raw checkpoint
    ├── model_traced.pt       # TorchScript — carries its own architecture
    └── model_meta.json       # "normalization_baked_in": false

load_model() prefers model_traced.pt, so the node runs standalone with nothing but torch installed. It falls back to best_model.pt, which needs NeuroIK's backend package importable.

#!/usr/bin/env python3
import rclpy
from rclpy.node import Node
from geometry_msgs.msg import Pose
from sensor_msgs.msg import JointState

class IKNode(Node):
    def __init__(self):
        super().__init__("neuroik_ik")
        self.sub = self.create_subscription(Pose, "target_pose", self.solve, 10)
        self.pub = self.create_publisher(JointState, "joint_command", 10)

    def solve(self, msg):
        pose = [msg.position.x, msg.position.y, msg.position.z, 0.0, 0.0, 0.0]
        joints = self.model_forward(pose)     # generated for you in ik_node.py

        out = JointState()
        out.header.stamp = self.get_clock().now().to_msg()
        out.name     = self.joint_names
        out.position = [float(j) for j in joints]
        self.pub.publish(out)

rclpy.init(); rclpy.spin(IKNode())

The generated node applies normalization explicitly -- its meta says normalization_baked_in: false. This is the one export where that is true.

7.10 TinyML (C header)

File: ik_model.h  ·  Runtime: ESP32, Arduino, STM32, any C99 toolchain  ·  Dependencies: none — just math.h

A single self-contained header: weights, normalization constants, and a real ik_forward(). No interpreter, no framework, no heap allocation.

SymbolPurpose
IK_INPUT_DIM / IK_OUTPUT_DIMPose and joint counts
ik_forward(pose, joints)Full inference — normalizes, runs the net, denormalizes
IK_X_MEAN, IK_X_STD, IK_Y_MEAN, IK_Y_STDConstants (already applied inside ik_forward)
#include "ik_model.h"
#include <stdio.h>

int main(void) {
    // Raw pose -- normalization is compiled into the generated function.
    float pose[6]   = {0.5f, 0.3f, 0.2f, 0.0f, 0.0f, 0.0f};
    float joints[N_JOINTS];

    ik_forward(pose, joints);

    for (int i = 0; i < N_JOINTS; i++)
        printf("q%d = %.3f\n", i + 1, joints[i]);
    return 0;
}

One header, no linking. Weights live in flash as const arrays; the only RAM used is a scratch buffer sized to the widest layer.

Warning: a complete ik_forward() is generated only for MLP models — a diffusion sampler or attention stack cannot be reduced to straight-line C. Other architectures still export their raw weights, but you must write the forward pass yourself. For embedded targets, train an MLP.
Tip: the header states its parameter count and estimated flash use at the top, and warns above ~50K parameters. A default 6→256→512→512→256→128→N MLP is roughly 564K parameters (~2.2 MB of flash) — fine on an ESP32 with room to spare, far too big for an Uno. Shrink the network for real MCU work.

7.11 Native library (.dll / .so)

Files: neuroik_ik.dll (or .so), ik_model.h, ik_dll.c, build_dll.bat, build_lib.sh  ·  Runtime: C/C++, C#/.NET, Python, LabVIEW, MATLAB  ·  Architectures: MLP, PINN

The same straight-line C that powers the TinyML export, compiled into a shared library with a plain C ABI. Zero dependencies beyond the C runtime — no PyTorch, no ONNX Runtime, no Python. Ideal when a native application needs IK and cannot take on an ML stack.

Exported symbolSignaturePurpose
neuroik_ik_forwardvoid (const float *pose, float *joints) Full inference — normalizes, runs the net, denormalizes
neuroik_input_dimint (void)Pose size (6)
neuroik_output_dimint (void)Joint count
#include <stdio.h>

// Exported by the generated library. C linkage -- no name mangling.
extern "C" int neuroik_ik_forward(const float* pose, float* joints);

int main() {
    float pose[6] = {0.5f, 0.3f, 0.2f, 0.0f, 0.0f, 0.0f};
    float joints[N_JOINTS];

    neuroik_ik_forward(pose, joints);     // returns 0 on success

    for (int i = 0; i < N_JOINTS; i++) printf("%.3f ", joints[i]);
}

Link neuroik.lib on Windows or -lneuroik on Linux. No runtime dependencies at all -- normalization is compiled in.

7.12 Core ML

File: NeuroIK.mlpackage (or NeuroIK.mlmodel) + model_meta.json  ·  Runtime: iOS 15+, macOS 12+, Apple Neural Engine  ·  Requires: pip install coremltools

For Apple platforms. Converted through TorchScript, so the same raw-pose-in / real-joints-out contract applies, and Core ML will schedule it on the Neural Engine where available. coremltools is an optional dependency — install it only if you need this format.

import CoreML

let model = try IKModel(configuration: MLModelConfiguration())

// Raw pose -- normalization is inside the model.
let input = try MLMultiArray(shape: [1, 6], dataType: .float32)
let pose: [Float] = [0.5, 0.3, 0.2, 0.0, 0.0, 0.0]
for (i, v) in pose.enumerated() { input[i] = NSNumber(value: v) }

let output = try model.prediction(pose: input)

for i in 0..<output.joints.count {
    print("q\(i + 1) = \(output.joints[i])")
}

Drag the .mlpackage into Xcode and it generates the IKModel class for you.

8Application Architecture

NeuroIK Desktop App
1
Robot Config & Recipe

Links, joint types, limits

2
Dataset Generator

Forward kinematics → reachable samples

3
ML Trainer

PyTorch · task-space loss

4
Simulation Studio

Three.js 3D · FK validation

Exporter

Trained checkpoint → deployable artifact

ONNX PyTorch TF Lite TensorRT ROS2 TinyML
Local storage
data/datasets data/models data/exports data/projects

Key internals

MechanismWhat it does
FK-first dataEvery sample is reachable by construction
Task-space lossTraining rewards reaching the target, not matching one arbitrary joint solution
Progress streamingGeneration and training push live progress to the UI
Output-dim matchingThe final layer matches the robot's joint count; a mismatched model is rejected rather than silently used
NormalizationPose and joints normalized by dataset statistics; constants saved with the checkpoint and folded into every export so deployed models take a raw pose and return real joint units
Export validationEach convertible format is run and compared against PyTorch numerically (not just by output shape) before the export is accepted
Degenerate maskingDimensions that never vary (a Delta's orientation) are masked so they add no noise
Diffusion sampling16 candidates → FK each → keep the closest
Rejection samplingPhysically impossible configurations (a Delta whose arms can't assemble) return NaN from FK and are resampled, never written to a dataset
Exact URDF engineImported robots use each joint's real origin/axis instead of generated DH parameters, so CAD geometry lines up with the motion

Kinematic engines

Each robot family has its own engine under backend/core/. They share one interface — forward_kinematics, inverse_kinematics, and a differentiable torch_forward_kinematics used for the task-space loss — so the dataset generator, trainer and simulator treat every robot identically.

EngineFKIK
scara.pyClosed-formAnalytical (elbow-down)
cartesian.pyDirect mappingDirect inverse
delta.pyTrilateration of three spheresPer-arm analytical, verified by FK round-trip
articulated.pyAuto-generated DH chainDamped least-squares, 12 restarts
urdf_robot.pyReal origin·axis chainDamped least-squares, 12 restarts

The DLS solver weights position 10× over orientation — reaching the XYZ target is the point of IK — and reports no solution rather than returning an unconverged guess.

9Project Management

Project dashboard
The Dashboard — counts, output folders, and the dataset/model selectors that drive retraining and export.

A project is the unit of work. It stores the robot type, recipe, every generated dataset, every training run with its metrics, and every export — as JSON under data/projects/. Reopening restores all of it, even after a restart.

Choosing which dataset / model to use

A project can hold several datasets and several trained models. The Dashboard has a dropdown for each:

  • Active dataset → used by ML Studio for training and retraining.
  • Active model → used by Simulation for prediction and by Export.

Deleting datasets and models

Each selector has a Delete button that removes the selected dataset or trained model from the project and deletes the app's working copy under data/. If you delete the active one, the next most recent is promoted automatically.

What is never deleted: copies mirrored into your own Project Root Folder, and models you already exported. Those live outside the app's storage — they're yours, so NeuroIK leaves them alone and tells you where the kept copy is.
Deletion is permanent. A deleted dataset can be regenerated from its recipe; a deleted model must be retrained.

Output folders

By default files stay in the app's data/ folder. Set a Project Root Folder (the Browse button opens a native folder picker) and NeuroIK also mirrors deliverables to:

<root>/<project name>/
├── datasets/          # dataset_<id>.csv
└── exports/
    ├── onnx/          # model.onnx + model_meta.json
    ├── tinyml/        # ik_model.h
    └── ros2/          # neuroik_ros2/ package

The copy in data/ is kept as the working copy the trainer and simulator read from.

Imported URDF projects

A project that uses a Custom URDF also stores the path to the source file. Reopening re-reads it, restoring the exact chain and the CAD meshes. If that folder has since moved or been deleted, the kinematics still come back from the recipe saved in the project — only the meshes are missing, and NeuroIK asks you to re-import to restore them.

10Troubleshooting & FAQ

1. The exported model returns nonsense values

Almost always double normalization. Every export except the ROS2 package now has the constants baked in, so you must feed a raw pose and read real joint units directly. Applying (pose - x_mean) / x_std yourself normalizes the input twice. Check normalization_baked_in in model_meta.json and see §7.1.

If you are loading the raw model.pt checkpoint, the opposite applies — that one is the bare network and does need the constants. Use TorchScript if you just want to run the model.

2. Export fails with “disagrees with PyTorch by …”

Working as intended: every convertible format is checked numerically against PyTorch before it is written, and the export is aborted rather than shipping a model that computes the wrong function. Report the number — it is expressed in units of a joint standard deviation.

3. “Only plain feed-forward models (MLP, PINN) can be converted”

TF Lite, Keras H5 and the native DLL rebuild the network layer by layer, which only works for a plain stack. Diffusion, GNN, transformer and RL keep their logic in forward(). Either train an MLP for those targets or export ONNX/TorchScript instead — see the support matrix.

4. Export succeeds but validation fails

The model's output width must equal the robot's joint count. Check that the active robot matches the one the model was trained on. For Core ML off macOS, and ROS2 when TorchScript tracing is unavailable, validation_passed is reported as false by design — the file is still produced, but the check could not be run here.

5. “No trained model found” when exporting

Train a model first. Export uses the model selected on the Dashboard, falling back to the most recent checkpoint.

6. Accuracy is poor

In order: generate more data (100K+), train longer, confirm the robot config matches training, then try a stronger architecture (Transformer/Diffusion). If you changed the recipe, you must regenerate and retrain.

7. Diffusion inference feels slow

By design — it samples 16 candidates and FK-checks each. For real-time control use MLP or Transformer.

8. TF Lite or Keras H5 export fails

TensorFlow is required for both: pip install tensorflow. Core ML likewise needs pip install coremltools. Every other format works without either.

9. TinyML header has no ik_forward()

You exported a non-MLP model. Only plain feed-forward networks reduce to straight-line C. Train an MLP.

10. The 3D viewport is blank

The viewport uses WebGL. Update your GPU drivers; on a remote/virtual desktop, hardware acceleration may be unavailable.

11. Dataset generation takes too long

1M samples with a complex chain takes several minutes. Use 100K while iterating.

12. "No .urdf or .xacro file found under: <path>"

The folder holds no description file. Point at the folder that actually contains the .urdf/.xacro (subfolders are searched too), or at the file itself. Quoted paths pasted from Explorer's Copy as path are handled.

13. My URDF imported, but a link's mesh is missing

Two likely reasons. Either the mesh is in an unsupported format — only STL, DAE and OBJ render; STEP/IGES show a placeholder — or the URDF never declares that link. NeuroIK draws what the description declares, so spare meshes shipped in the folder but not referenced by any <link> won't appear. The import warnings list both cases.

14. "This file uses <xacro:macro>"

Macro expansion isn't supported — a partial implementation could silently build the wrong robot. Pre-expand with the ROS tool and import the result:

Bash
xacro robot.xacro -o robot.urdf

15. My imported robot has one DOF fewer than I expected

Probably a <mimic> joint — a gripper's second finger, say. It is mechanically driven by another joint, so it is not an independent DOF and must not be a network output. It still moves in the 3D view, following its source joint.