When introducing hand-eye calibration for robot vision, it is important to distinguish between two separate acceptance criteria: the ability to visually detect the workpiece in the camera image, and the robot’s ability to reliably reach that position. This distinction is especially critical in Thai factories, where multiple equipment manufacturers and system integrators may be involved. Before issuing a purchase order, you must decide: “What is the reference coordinate system?”, “In which poses will data be collected?”, and “With which workpieces will pass/fail be judged?” This article organizes the calibration mechanism, error isolation, and re-calibration procedures for operation, so that implementation managers can translate them into RFP, FAT, and SAT requirements. This is not an article that promises generic accuracy or ROI values for specific models.
The First Decision: Will the Camera Move or Be Fixed?
“Hand-eye” is a general term for determining the coordinate relationship between a robot and a camera. In eye-in-hand configurations, the camera is mounted on the robot wrist or tool and moves together with the robot. In eye-to-hand setups, the camera is fixed to the equipment, observing both the workpiece and robot externally. These two approaches differ in jig design, cabling, imaging poses, calibration target mounting, and re-calibration procedures. The official OpenCV documentation also explains the transformation matrices for both configurations separately. OpenCV Hand-Eye Calibration Documentation
Wrist mounting (eye-in-hand) is suitable when the robot needs to view the same object from multiple positions, approach while avoiding obstacles, or dynamically use the camera’s field of view. However, camera brackets, cable bending, and the repeatability of mounting during tool changes can affect the coordinate relationship. Fixed cameras make it easier to observe multiple workpieces or conveyor surroundings within the field of view, but if the installation structure or the relative position to the robot base changes, the transformation must be reviewed. Cognex’s official VGR procedures also distinguish between robot-mounted and fixed camera methods. Cognex VGR Procedure
After narrowing down your approach by reading the overall guide to robot vision implementation, this article will focus specifically on coordinate system alignment and acceptance testing. If your main topic is moving workpieces or depth cameras, also refer to 3D Vision and Conveyor Tracking, as those introduce additional challenges such as tracking and time synchronization, making calibration with only stationary workpieces insufficient.
What Does Hand-Eye Calibration Actually Determine?
The robot controller manages coordinates for the base, flange, and tool center point (TCP). The camera side has image coordinates, camera coordinates, and calibration board coordinates. For picking, the position and orientation of the workpiece found in the image must be converted into robot-operable coordinates. For example, in eye-in-hand, you determine the rigid transformation from the camera to the tool, combining it with the flange pose reported by the robot. In eye-to-hand, you determine the relationship between the fixed camera and the robot base. If you confuse the orientation of these frames, the math may work, but the robot will be commanded to the wrong position in practice.
To determine this transformation, you capture images of a calibration target of known dimensions from multiple robot poses, associating each image with the corresponding robot pose. OpenCV’s calibrateHandEye takes gripper→base and target→camera as inputs and outputs camera→gripper. For eye-to-hand, the input transformations are arranged differently, so implementers must fix frame names, arrow directions, units, and rotation representations with diagrams and test data. OpenCV requires movements that include non-parallel rotation axes to solve for the transformation, recommending at least three different poses, and in practice, more. This does not mean that simply increasing the number of poses guarantees success. OpenCV Camera Calibration

“Known dimensions” of the target do not refer only to the design values on the print file. Physical dimensions and flatness can be affected by printing, adhesion, board warping, and fixture flex, which in turn affect the position estimation from images. The purchase specification should include the target type, measured dimensions, identification ID, fixation method, and procedures for re-verification upon replacement. Universal Robots’ procedures also specify that the calibration board must not be moved during calibration and should be evenly illuminated to avoid shadows. UR Camera Calibration Procedure
Do Not Confuse Camera Intrinsics, TCP, and Hand-Eye Transform
If you lump three different tasks under the single word “calibration,” it becomes unclear who is responsible when issues arise during FAT. Camera intrinsics refer to focal length, principal point, lens distortion, and other parameters relating images to light rays. TCP is the position and orientation from the robot flange to the actual working point. Hand-eye transformation is the external relationship linking the robot’s frame and the camera’s frame. If you adjust one to mask errors in another, the error will reappear as soon as you change the verification position.
OpenCV’s camera calibration API determines intrinsic and extrinsic parameters from multiple pattern views and returns the reprojection error. However, the reprojection error only indicates image-level consistency and does not directly guarantee the actual positional error at the gripping point. OpenCV calib3d Documentation Meanwhile, Universal Robots’ TCP documentation defines TCP as a translation and rotation relative to the flange, and shows that the active TCP is used for operation. Even if only a different TCP is selected after a tool change, the working point will shift, even if the image side is correct. UR TCP Configuration Documentation
During implementation, save all three as separate files and versions, recording “who,” “when,” and “with which jig” each was obtained. When switching recipes, ensure the currently active combination of camera intrinsics, TCP, and hand-eye transformation is displayed on the screen. It is also important to log frame names and units. If you only record “x=0.15,” but do not specify whether it is in meters or millimeters, or whether it is in base or camera coordinates, it cannot be used for troubleshooting.
Robot Vision Calibration Implementation Flow: From On-Site Confirmation to Acceptance
1. Define Workpieces and Processes First
Before selecting a camera model, gather representative and exceptional workpieces. Record material, gloss, color, dimensional tolerances, placement, stacking, orientation, variation in posture, supply tray or conveyor position. Clearly state whether the target coordinate is the center, a hole, a corner, or a gripping surface. Just because a workpiece is “recognized” does not guarantee that a suction pad or gripper can safely engage. Agree with process owners on which process losses are caused by grip failures or collisions.
In Thai factories, production lighting, work shifts, reflections from surrounding equipment, and cleaning methods may differ from lab conditions. Rather than generalizing from market statistics, collect conditions from the actual line. Attach not only photos of representative workpieces to the RFP, but also samples of defective or difficult conditions and the environmental conditions at the time of sampling. Difficult conditions not shown to developers are likely to be judged as “out of specification” after mass production.
2. Freeze Coordinate Diagrams and Mechanical References
In the drawings, define each frame—robot base, flange, TCP, camera, target, fixture, workpiece—only once. For fixed cameras, specify the relationship between the camera support and base installation; for wrist mounting, specify the bracket-to-flange relationship. Define transformation arrows in text, such as “transform a point in A coordinates to B coordinates,” and do not leave matrix naming to the discretion of designers. Since robot manufacturers differ in pose representation, axis orientation, and angle units, standardize these at the implementation input boundary.
Mechanical home positions, stoppers, locating pins, and repeatability of jig exchange are also prerequisites for calibration. Even if you upgrade only the camera, if trays are placed differently each time, detection and gripping yields will not improve. Any change to the mechanical reference should be treated as requiring re-verification of the hand-eye transformation and working point. After maintenance staff retighten the support frame on site, provide a simple procedure for position confirmation.
3. Capture Images and Perform Intrinsic Calibration
Collecting many images where the target only appears in one corner of the field of view is insufficient to check for overall distortion or working range. Capture images at multiple positions within the expected working distance and field of view, checking focus, exposure, lighting, reflections, and motion blur. After mounting the camera and lens, fix the settings and check whether autofocus or auto-exposure changes affect repeatability. OpenCV’s reprojection error is a useful indicator for image processing quality, but do not use this value alone to judge mechanical positioning pass/fail.
In environments where lighting changes by shift, check target and workpiece detection under representative morning, daytime, and nighttime conditions. If lighting is enclosed within the equipment, sensitivity to external disturbances can be minimized, but on lines with ambient light or reflective objects, record imaging conditions in both FAT and SAT. Since image data may include personal or confidential items, define the scope, retention period, and access rights for storage in the implementation specification.
4. Finalize TCP and Imaging Poses
Teach the TCP using the actual gripper, suction device, or jig to be used. If you only adjust hand-eye with a temporary TCP, the gripping point will change after the final tool is installed. UR’s documentation explains that TCP is managed as a position and orientation relative to the flange, and the active TCP is reflected in operation. Therefore, during FAT, record the TCP name and version on the acceptance record. UR TCP Configuration Documentation
For imaging poses, simply moving the same point slightly sideways is not enough. Design poses that cover the working area and avoid bias in rotation axes. Simultaneously check that the target does not leave the field of view, the robot does not interfere, and camera cables are not pulled. Apply safety fence and collaborative operation risk assessments to manual operation and teaching modes used for calibration. Decide on an operation policy that does not disable safety devices even as an exception during calibration.
5. Calculate Hand-Eye and Verify with Unused Poses
Even if you get good results with the poses used for calculation, you cannot confirm generalization to actual workpieces. Re-measure the target or reference point at robot poses and field positions not used for estimation. Compare the predicted gripping point from the camera with an independent physical reference, recording both position and angle. Record not only the “average” measurement, but also the maximum, distribution, and edge tendencies. Not accepting a single impressive demo as a pass is the most effective risk reduction for procurement.
Save the calculation results together with the transformation matrix, input robot poses, target detection results, excluded images, and software version. If recalculation later produces different results, you can trace which input or algorithm change was responsible. Product-specific imaging procedures such as Cognex’s may specify recommended numbers of poses, but do not blindly apply these to other cameras or robots—instead, use the procedures and acceptance data for the products you adopt.

How to Specify “Accuracy in mm” in Requirements
If you write only “accuracy ±○mm” in the RFP, it is unclear whether this refers to image detection error, TCP position error, robot repeatability, jig variation, or workpiece individual differences. What is needed is to first define the final gripping, insertion, or assembly error tolerable by the process, and attach a verification method. For example, for insertion into a hole, the clearance between the hole and pin, proximity to other parts or safety clearances, and workpiece tilt are important. For box picking, the gripping surface area and retention during transport are important. Even with the same camera specification, process tolerances differ.
Be cautious about simply adding up errors and displaying a “guaranteed accuracy.” Independent variations and systematic shifts due to temperature or installation are fundamentally different. First, break down the causes into “image detection,” “image to 3D pose,” “hand-eye transformation,” “TCP,” “robot/mechanics,” and “workpiece supply,” and define the observed quantities at each stage. If the image reprojection error is small but the actual gripping position is off, suspect target dimensions, transformation direction, TCP, or installation. Conversely, if image detection is unstable, simply reteaching the robot will not address the root cause.
Acceptance values are determined from the product drawings and process capability for each project and agreed with the supplier. This article does not provide generic pass/fail mm values. Set measurement points not only at the center of the field of view, but also at the edges, upper and lower workpiece heights, and before and after continuous operation, to help uncover problems masked by local corrections. For pass/fail judgment, also define the number of test runs, handling of NGs, retest conditions, and management of measuring instruments and reference jigs. If only the numbers are agreed and the measurement method is undecided, problems will carry over from FAT to SAT.
| Error Layer | What Is Observed On-Site | Main Verification Method | Troubleshooting When Failing |
|---|---|---|---|
| Imaging/Detection | Target corners, workpiece features, exposure state | Saved images and detection logs, field position checks | Lighting, focus, reflection, lens, algorithm |
| Camera Calibration | Reprojection error, residual distortion | Unused images and working range checks | Board dimensions, flatness, image bias |
| Hand-Eye | Predicted reference point coordinates/angles | Unused poses and independent reference jigs | Transformation direction, pose distribution, board fixation |
| TCP/Robot | Actual tool tip arrival position | Check with actual tool and measuring jig | TCP selection, tool mounting, machine home |
| Process | Success/failure of gripping/insertion/transport | Continuous tests with representative and difficult workpieces | Workpiece supply, interference, tolerance |
The items in this table serve as a template for RFP measurement planning. The units and meanings of image reprojection error and process success rate are different. Do not convert between them and call them “the same accuracy”; record both.
What to Include in the RFP: What to Provide to Suppliers
To compare bidders, you must provide the same conditions. First, prepare process videos, workpiece drawings, actual samples, jig drawings, robot model/control software, available I/O or communication methods, installation space, lighting, and safety constraints. Next, specify your preference and rationale for eye-in-hand or eye-to-hand, the scope of changes allowed to existing equipment, and your policy for ownership and backup of calibration data on site. If you cannot decide on a method, request both options and have suppliers submit the required jigs, downtime, and maintenance work for each, in a format that allows comparison.
As deliverables, specify coordinate system diagrams, calibration procedures, target specifications, version-controlled parameters, logs of images and robot poses, state transitions for exception handling, FAT/SAT test sheets, and re-calibration procedures for maintenance staff. If source code, licensing, or learning model update methods are involved, specify these separately. Simply seeing “calibration successful” on the screen does not tell the operator which calibration set was applied to the production recipe. It is desirable to display the active setting ID and design the system so that operation cannot start if the wrong set is selected.
Include in your RFP questions such as “What changes require re-calibration?”, “Who and how will verify calibration validity?”, and “What are the procedures and required items for recovery after camera replacement?” If the proposal only presents a single accuracy value, additionally confirm the evaluation workpiece, field position, number of test runs, measurement method, and surrounding conditions. If the camera and robot are supplied by different companies, also clarify responsibility for communication timestamps, pose reading timing, transformation implementation, and log acquisition in case of failure.
What to Check in FAT: Going Beyond “It Works” Before Factory Shipment
FAT is performed at the supplier’s site, but should reproduce mass production conditions as closely as possible. First, check the actual configuration of machine installation, camera/lens/lighting/cabling, target, tool, and TCP, and cross-check with drawings and version numbers. Then, delete or archive the calibration data and confirm that re-calibration can be performed using only the procedure manual—this reveals reliance on tacit knowledge. Depending on the configuration, data deletion may be risky, so confirm backup and restoration procedures in advance.
In testing, include positions and poses not used for learning/calibration, representative workpieces with different sizes, colors, and reflections, and edge supply positions. Record not only recognition success rate, but also whether the robot gripped the workpiece, dropped it during transport, picked the wrong workpiece, or transitioned to safe stop/retry/ejection when out of view or undetected. If you link evaluation results—images, detected coordinates, robot poses, actual arrival positions, OK/NG, and occurrence conditions—under the same trial ID, reproduction becomes easier.
Lock the set of parameters that passed FAT with a hash value or version number, and compare them during SAT after transport and reinstallation. Deliver calibration boards and jigs with the equipment, or provide drawings for local remanufacture. Also test abnormal cases such as power recovery, communication loss, camera disconnection, lighting failure, gripper replacement, and incorrect TCP selection. FAT is not complete at the moment calibration calculation succeeds, but when the equipment can safely recover as a system.
What to Check in SAT: Addressing Installation and Variability in Thailand
In SAT, check the positions of the robot base, camera mount, jigs, and lighting after installation against the FAT drawings. If you cannot rule out the possibility that the bracket moved during transport or wiring, recalibrate on site. Universal Robots’ camera mounting documentation requires repeating calibration after adjusting camera position. While this is a manufacturer-specific procedure, it also reflects the basic principle that the transformation depends on rigid body placement. UR Camera Mounting Documentation
On site, use actual products and supply equipment, and select representative conditions for planned shifts, lighting, post-cleaning, and temperature changes. If you cannot reproduce all environments in a short time, separate immediate SAT acceptance criteria from post-mass production observation items, and assign responsible persons and deadlines. For Thai-speaking operators, train them using actual screen displays on the meaning of error screens, retry limits, calibration-required warnings, and maintenance contacts. Even if an English manual exists, judgment may differ by shift.
If there is a discrepancy between FAT and SAT, first check for differences in imaging conditions and machine layout, then sequentially isolate TCP, transformation parameters, and workpiece supply. Before temporarily passing by adding correction values to coordinates on site, save the original data. If you add corrections, specify the scope, reference temperature, recipe, and cancellation method, and ensure that double corrections do not occur at the next recalibration. The goal is not to prohibit on-site fine-tuning, but to ensure traceability of causes and versions.

Drift and Re-Calibration: What Should Be Monitored After Mass Production?
Calibration values are not permanent constants. If the camera or lens is replaced, the bracket loosens, the robot or jig is relocated, the tool is changed, a collision occurs, or the reference board is damaged, check the affected frames. If camera internal settings are changed, perform intrinsic calibration; if the working point from the flange changes, update the TCP; if the relative arrangement of camera and robot changes, redo hand-eye calibration. Repeating all steps every time increases downtime, but without a change history, you cannot judge what can be skipped.
For daily checks, use a fixed reference workpiece or check jig to compare the detected position in the image with the robot’s predicted working point—a “health check.” If the result deviates from the reference, stop mass production, isolate the cause, and proceed to the formal calibration procedure. Set this threshold on site based on process tolerances and measurement variation; do not use generic values from articles. Save inspection records including parameter version, TCP used, camera settings, jig ID, date/time, operator, and judgment result.
Also define calibration authority. Operators detect anomalies and stop/notify; maintenance staff check machine and wiring status; engineers update and verify parameters; managers approve re-entry into mass production. To avoid “it’s been drifting since yesterday” being passed on only verbally at shift change, make inspection trends visible as graphs or logs. However, sometimes it is faster to check the jig or tool replacement history before statistically estimating the cause of drift.
Common Implementation Failures and How to Prevent Them
First, reversing the direction of coordinate transformations. If you treat camera→tool and tool→camera as the same, and mix units, it may look plausible for limited poses but fail at the edges. Prevent this with transformation diagrams showing frame names, single-point tests at known positions, and explicit implementation reviews for matrix inversion. Second, capturing images in different poses while moving the target. If the calibration method assumes a fixed target, you must redo data collection if the target is touched during acquisition.
Third, judging “robot accuracy” based only on image reprojection error. Reprojection error is an evaluation metric for the camera model; process success must be evaluated separately with the actual tool and workpiece. Fourth, failing to track TCP selection from the screen, logs, or recipes. The more tools and TCPs on the same robot, the more important it is to record the active value. Fifth, saving calibration data only inside the equipment and not delivering a backup. If a failure occurs, you cannot reproduce the same conditions, delaying recovery.
Sixth, repeatedly testing only the same central workpiece in FAT, then encountering problems at the field edge or with reflective workpieces in SAT. Specify spatial distribution of evaluation points and difficult workpiece conditions in the test sheet. Seventh, confusing communication delays or moving workpieces with hand-eye calibration issues. For moving conveyors, separately verify imaging time, encoder, and robot command timing. If you succeed with stationary workpieces, the discussion in 3D Vision and Conveyor Tracking will be helpful. Avoid lumping all problems under “the camera is off”—this shortens recovery time.
Implementation Decision Checklist
Before ordering, confirm: ① samples of workpieces and difficult conditions, ② required final process accuracy and measurement method, ③ rationale for eye-in-hand or eye-to-hand, ④ coordinate diagrams, ⑤ responsibility boundaries for camera intrinsics, TCP, and hand-eye, ⑥ FAT/SAT including unused poses, ⑦ triggers and local responsibility for recalibration, ⑧ parameter versioning and backup, ⑨ safe stop and recovery in case of anomalies, and ⑩ procedures readable by Thai operators. In proposal comparison tables, separate “possible” from “proven on actual equipment” to avoid misreading apparent cost differences.
If starting from minimal proof, include not only one representative workpiece but also a few difficult cases, use actual jigs and tools, and measure position error and process success in unused poses. If successful, gradually expand workpiece types and operating conditions. Even before issuing an RFP, with photos of workpieces and processes, tolerances, expected cycle time, and existing equipment constraints, you can identify missing information. Rather than starting with a “manufacturer feature list,” work backward from process acceptance criteria to make FAT/SAT discussions more concrete.
Frequently Asked Questions
What is hand-eye calibration in robot vision?
It is the calibration that determines the relationship between the camera and robot coordinate systems, allowing positions and orientations found in images to be converted into robot-usable coordinates. The required transformation and target placement differ between eye-in-hand and eye-to-hand. Camera intrinsic calibration and TCP setting are also required separately; process accuracy is not determined by hand-eye transformation alone.
How many imaging poses are sufficient?
There is no universal number for all projects. OpenCV explains the minimum conditions for solving and recommends using more poses, but what matters is diversity in rotation axes, working range, target appearance, and verification with unused poses. Decide based on the procedures for your chosen camera product and acceptance testing.
Is recalibration always necessary after camera replacement?
Lens or camera internal characteristics and mounting position may change. After replacement, check camera intrinsics and the relative arrangement of camera and robot, and recalibrate the affected scope. Even if no effect is seen in pre/post health checks, it is important to document the acceptance procedure and results.
If FAT passes, can SAT in the Thai factory be omitted?
Installation, lighting, jigs, supply equipment, and actual workpieces differ from the FAT environment. At minimum, SAT must verify changes and accept the actual equipment. Which tests to repeat depends on the scope of transport/reinstallation and RFP acceptance criteria. FAT results alone cannot guarantee local mass production processes.
Why does gripping deviate even after successful hand-eye calibration?
Possible causes include image detection, intrinsic calibration, target dimensions, transformation direction, TCP selection, tool mounting, jig position, and workpiece supply. By separately measuring image reprojection error, reference point error in unused poses, and actual tool arrival error, you can narrow down the cause. For moving workpieces, also check time synchronization and tracking control.
Conclusion
For successful implementation of robot vision hand-eye calibration, it is essential to first align process acceptance criteria and coordinate system definitions, rather than focusing solely on method selection. Manage camera intrinsics, TCP, and hand-eye transformation separately, and verify with unused poses and actual workpieces. Confirm repeatability and abnormal recovery in FAT, check installation conditions in SAT, and track drift during mass production with change history and health checks. Rather than relying on generic accuracy numbers, the measurement method and responsibility boundaries suited to your process will determine success.
At TOMAS TECH, we can assist with coordinate system diagrams, RFP evaluation items, and FAT/SAT test sheet organization, tailored to your workpieces, jigs, and robot configuration in Thai factories. Even at the consideration stage before method selection, please contact us with your process requirements.
References
- OpenCV Camera Calibration: calibrateHandEye — Hand-eye input/output transformations, eye-in-hand/eye-to-hand, pose conditions.
- OpenCV Camera Calibration and 3D Reconstruction — Intrinsic calibration and reprojection error.
- Universal Robots: Camera calibration — Board fixation and lighting conditions. UR product-specific procedures.
- Universal Robots: TCP Configuration — Definition of TCP and active TCP.
- Universal Robots: Attaching the Camera Mount — Recalibration after changing camera position.
- Cognex: VGR Hand-Eye Calibration — Fixed and wrist-mounted configurations, product-specific procedures.