Programming & tutorials
Robotics Programming: From First Code to Sensor Feedback
Start robotics programming with Python, ROS 2 and sensor feedback. Follow a hardware-free tactile data exercise, then explore messages, replay and robot learning.
Robotics programming connects observations to decisions, control and fresh feedback. You can learn its data and communication foundations on a laptop before working with a robot.
The Python project below has been run in a clean environment. The ROS 2 walkthrough identifies its source checks, upstream build evidence and unexecuted runtime steps separately.
What is robotics programming?
Robotics programming is the work of turning a task into software that senses the world, estimates the current state, chooses what to do, and checks the result. A fixed sequence can be useful, but a physical robot also has to deal with uncertain measurements and changing conditions.
Think of a gripper picking up a cup. Perception locates the cup; state processing combines observations with the arm position; planning chooses an approach; control tracks a target; feedback reveals whether contact happened. A failed observation, stale timestamp or unexpected force can change the next decision. This tutorial path starts with that observation and feedback layer.
| Stage | Programming task | First exercise |
|---|---|---|
| Perception | Read and interpret sensors. | Load values, units and validity flags from a CSV. |
| State processing | Associate measurements with time and geometry. | Validate timestamps and reconstruct the taxel grid. |
| Planning | Choose a path or task sequence under constraints. | Study a planner in a documented simulator after learning the data interface. |
| Control | Track targets using repeated feedback. | Learn the difference between a target, an actuator command and an observation. |
| Feedback | Compare what happened with what was expected. | Inspect threshold events and replay the observations that caused them. |
Python or C++: choose by the work
Python is a practical starting point for data inspection, plots, tests, research tooling and many ROS 2 nodes. Basic functions, lists, files, exceptions and virtual environments are enough for the Python exercise here.
C++ is common in hardware interfaces, performance-sensitive processing and controllers. It offers more explicit control over memory and execution costs, at the price of a more demanding build and debugging workflow. A Python program may also call compiled numerical libraries.
A language choice is not a real-time guarantee. Scheduling, middleware, allocations, operating-system configuration and worst-case execution time all affect deadlines. Measure the complete system against its requirements before making a timing claim.
Know which layer each tool belongs to
A useful robot programming stack contains several kinds of tools. They complement each other; installing a middleware package does not provide a robot model, motion planner or trained policy.
| Category | Examples | Responsibility |
|---|---|---|
| Languages | Python, C++ | Express algorithms, application logic and interfaces. |
| Middleware | ROS 2 | Connect nodes with typed messages, discovery, services and actions; provide inspection tools. |
| Simulation | Gazebo | Model environments and robot dynamics with stated approximations; simulated results still need physical validation. |
| Motion planning | MoveIt 2 | Plan movements with robot geometry, kinematics and collision constraints. |
| Learning tools | PyTorch and task-specific robot-learning libraries | Train and evaluate representations or policies against documented data and action spaces. |
| Recording and inspection | rosbag2, CSV tools, Matplotlib | Preserve observations, examine failures and compare reproducible runs. |
A learning route without a robot
Begin with files: read a small dataset, validate its shape and draw an output you can inspect. Next, learn the same concepts in a stream: publish synthetic messages, monitor them and record them. Then stop the publisher and replay the recording so the inputs can be examined again.
Synthetic messages test software plumbing. They do not simulate a physical sensor or prove a robot can perform a task. When you need geometry and dynamics, follow an official simulator tutorial with its own supported ROS and simulator versions.
- First: finish the Python exercise and explain every unknown value in its output.
- Next: use the ROS 2 walkthrough on its declared Linux/ROS combination and record your own runtime evidence.
- Then: connect observations to robot-learning concepts, dataset access conditions and evaluation splits before training a model.
Work with Robot Data
Start with the Python CSV project: inspect missing values, generate plots and explain the resulting contact events. Then run the LeRobot numeric checker to understand how episodes, timestamps and state/action dimensions fit together. Both exercises include synthetic data, actual outputs and behavior tests; neither needs a robot, tactile sensor, ROS installation or GPU.
When preparing your own data, use the collection guide to plan demonstrations, synchronization and failure labels. Compare original dataset fields and access terms before adapting a recording for a particular learning policy.
Add Tactile Feedback
Begin with a quantity you can interpret: an image baseline, reconstructed depth and estimated force require different references. The DIGIT and GelSight Mini calibration guide explains those boundaries and provides a blank recording template; it is source-based guidance, not a hardware experiment.
Use the Python exercise to preserve missing values and inspect synthetic contact events. Then follow the ROS 2 message workflow to understand transport and replay before integrating a sensor. The ROS runtime steps remain explicitly unverified locally.
From a signal to tactile AI and robot learning
Once a sensor stream has stable units, timestamps, geometry and validity semantics, it can support features such as contact location or changes in a tactile pattern. Those features may become observations for manipulation, supervision for representation learning, or evaluation signals.
The threshold exercise here is a transparent teaching rule, not a trained tactile model or calibrated contact detector. Real tactile AI needs sensor-specific calibration, synchronized robot observations, suitable labels, held-out data and a stated task. A dataset directory helps find candidates, but each original dataset defines its own fields and license.
Common questions
What should I learn first?
Start with Python functions, sequences, file handling, exceptions and basic tests. Then practice timestamps, coordinate systems and missing-data handling before adding ROS 2 nodes.
What environment do I need?
The Python exercise was verified with CPython 3.13.3 and Matplotlib 3.10.6 on Windows x64. The ROS walkthrough targets Ubuntu 26.04 with ROS 2 Lyrical; it is a separate setup with a different verification scope.
Can I start programming robots without a robot?
Yes. Data processing, synthetic messages, recording and replay teach useful interfaces without hardware. They do not establish hardware compatibility or physical task performance.
Do I need machine learning for every robot program?
No. File processing, state machines, planning and feedback can use explicit rules. Add learning when you have a suitable task, data and evaluation protocol.
Next steps
- Process synthetic tactile data with Python →
Run a CSV quality check, generate plots and export teaching contact events.
- Publish and replay tactile messages in ROS 2 →
Source-checked starter-kit walkthrough; local ROS runtime validation is pending.
- Robot learning →
Understand demonstrations, policies and evaluation.