G6D

G6D: Geometric Learning-Free RGB-D 6D Pose Solver for Robotic Manipulation

Yixuan Liang1, William Chen2, Yunan Wang1, Jizhou Yan1, Zhao Jin1, Changling Liu2, and Chuxiong Hu1,*

1 Tsinghua University2 Sapient Intelligence

* Corresponding author

Paper (arXiv) Watch video Code (G6D) Code (G6D-CPU) BibTeX
Project video2:59 1080p

Abstract

6D object pose estimation is a key component of robotic manipulation and automation. Recent zero-shot methods have improved generalization to unseen objects, but many still rely on large pretrained models, limiting interpretability and deployment flexibility when perception must share computational resources with planning and control. We present G6D, a learning-free, geometry-driven RGB-D pose solver for robotic manipulation. Given an RGB-D observation, an object mask, camera intrinsics, and a textured CAD model, G6D builds reusable metric correspondence templates offline and establishes ORB-based 2D–3D matches online. MSAC-PnP generates candidate poses, mask coverage and intersection-over-union select plausible hypotheses, and trimmed depth ICP aligns their surfaces to the observation. Multi-hypothesis pose optimization then jointly refines soft-mask and depth consistency through differentiable rasterization, retaining the best physical-score checkpoint. An optional confidence-triggered geometric post-refinement explores complementary hypotheses when needed. The pipeline requires no learned pose network or object-specific pose training and exposes geometric intermediate results throughout inference. Evaluations on LineMOD and five BOP datasets, together with a real-world robotic tabletop-tidying experiment, examine its estimation capability and practical use as a pose solver. Code and reusable examples are available in the project repository.

Method Overview

Prepare once. Estimate geometrically.

Given an RGB-D observation, an object mask, a CAD model, and camera intrinsics, G6D recovers the object's rotation and translation. Every intermediate estimate is an explicit 3D pose—no learned pose network is required.

A reusable template bank connects image features to metric CAD points. Online, appearance initializes the pose; geometric evidence selects and refines it.
Open the full-resolution framework ↗
OFFLINE

Build metric correspondence templates. Render 324 views per CAD model—162 viewing directions with two orthogonal in-plane rotations. Extract ORB descriptors and use the known rendering depth and camera pose to associate each descriptor with its 3D model point. Cache this bank for reuse.

  1. 01

    Connect pixels to the model

    Use a square crop and uniform resizing to preserve shape. Match the input ORB descriptors to the template bank using Hamming distance. A match-quality score retains 256 correspondence sets for MSAC-PnP, which converts their 2D–3D matches into candidate poses.

    RGB-D + mask → 2D–3D matches → pose candidates
  2. 02

    Select plausible poses, then align depth

    Render each candidate and check its target-mask coverage and intersection-over-union. Keep 18 hypotheses. Trim unreliable point pairs and use rigid ICP updates to bring each visible model surface toward the measured depth.

    Mask consistency → 18 hypotheses → depth alignment
  3. 03

    Refine multiple hypotheses together

    Multi-hypothesis pose optimization (MHPO) uses differentiable rasterization and Adam to optimize soft-mask and depth consistency. Retain the best physical-score checkpoint across hypotheses and iterations, rather than simply returning the last iterate.

    Soft mask + metric depth → best object-to-camera pose

Optional geometric post-refinement. When the primary estimate remains uncertain, complementary depth and appearance proposals can trigger another geometric refinement. Replace the current output only if its physical consistency score improves sufficiently.

Input RGB-D · mask · CAD · KOutput T ∈ SE(3)Learning No pose-network training

Qualitative Results

Watch the pose converge, one object at a time. Explore 54 selected examples across six datasets, from cluttered household scenes to texture-poor industrial parts.

Input→PnP→Depth ICP→MHPO→Output

Each clip loops through the same estimation sequence. Use the arrows to browse; select a clip to inspect it at full size.

Real-World Robotic Deployment

G6D serves as the pose solver in a robotic pick-and-place pipeline. A Franka robot localizes, grasps, and stores five tabletop objects in sequence, turning geometric estimates into a practical tidying task.

BibTeX

arXiv:2609.23566
@misc{liang2026g6d,
  title         = {{G6D}: Geometric Learning-Free {RGB-D} {6D} Pose Solver for Robotic Manipulation},
  author        = {Liang, Yixuan and Chen, William and Wang, Yunan and Yan, Jizhou and Jin, Zhao and Liu, Changling and Hu, Chuxiong},
  year          = {2026},
  eprint        = {2609.23566},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2609.23566}
}