Repository navigation
Add shared scene services and batched geometry kernels - #761
Conversation
|
Scene Randomization Design: Persistence and PolicyHi Yueci! Thanks for your work on the shared scene services in #761. I've drafted the proposal below to build on those services, and I'd appreciate your feedback on whether it aligns with the intended scope. Status: design draft. Base-data caching, randomization adapters, and layout I/O remain to be implemented. Date: 2026-10-13. The proposed approach groups each base object directly supported by the table with all objects supported above it, and randomizes the group using its combined 2D AABB. The source scene and assets are shared; each result stores only the group layout. Persistent DataWhat to Store
The base YAML and layout JSON files are new GenSim-derived data defined by this design. The proposed directory structure is below. Entries marked "new" belong to this design: The source format remains Base DataThe first version consumes rigid-body support subtrees on a single horizontal, connected table schema_version: 1
coordinate_frame: local_arena_z_up
source_scene:
scene_id: scene-engine-example
export_dir: .
content_sha256: "0000000000000000000000000000000000000000000000000000000000000000"
table:
entity_id: table
center_xy: [0.0, 0.0]
size_xy: [1.2, 0.8]
surface_z: 0.75
groups:
- root_id: bottle_001
root_pose:
position: [-0.30, 0.10, 0.85]
quaternion_xyzw: [0.0, 0.0, 0.0, 1.0]
members: []
aabb_2d:
center_xy: [-0.30, 0.10]
size_xy: [0.10, 0.10]
- root_id: tray_001
root_pose:
position: [0.15, 0.10, 0.77]
quaternion_xyzw: [0.0, 0.0, 0.0, 1.0]
members:
- entity_id: block_001
parent_id: cup_001
pose_in_root:
position: [0.03, 0.02, 0.15]
quaternion_xyzw: [0.0, 0.0, 0.0, 1.0]
- entity_id: cup_001
parent_id: tray_001
pose_in_root:
position: [0.03, 0.02, 0.08]
quaternion_xyzw: [0.0, 0.0, 0.0, 1.0]
aabb_2d:
center_xy: [0.16, 0.10]
size_xy: [0.30, 0.20]
AABB minimum and maximum corners are derived from Layouts and Run RecordsEach candidate layout has one JSON file containing all groups; a run stores multiple candidate {
"schema_version": 1,
"candidate_id": "candidate_000001",
"proposal_ordinal": 1,
"seed": 42,
"algorithm_version": "group_xy_slsqp/v1",
"coordinate_frame": "local_arena_z_up",
"geometry_model": "fixed_orientation_aabb",
"source_scene": {
"scene_id": "scene-engine-example",
"content_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
"base_data": "../../existing_scene/scene_export/scene_randomization_base.yaml",
"base_data_sha256": "0000000000000000000000000000000000000000000000000000000000000000"
},
"policy_file": "../policy_resolved.yaml",
"policy_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
"groups": [
{"root_id": "bottle_001", "center_xy": [-0.30, 0.10], "yaw_delta_deg": 0.0},
{"root_id": "tray_001", "center_xy": [0.36, 0.10], "yaw_delta_deg": 0.0}
]
}
Every valid candidate is saved as a layout JSON; the number of files equals the actual number File references are resolved relative to the file that contains them. The source Preparing and Reusing Base Data
Table detection currently retains only the largest outer contour, which alone does not prove Changes to entities, support relations, member poses, geometry, scale, or the table region require Randomization PolicyFirst-Version Scope Aligned with the PRPR #761 initially supports fixed-orientation XY/AABB layouts on horizontal supports. GenSim owns grouping, coordinate conversion, and format adaptation, and directly reuses ConfigurationThis example generates 100 valid candidates and then selects 20: schema_version: 3
input:
base_data: ./existing_scene/scene_export/scene_randomization_base.yaml
groups:
tray_001:
mode: sample
translation_xy_m:
x: null
y: null
bottle_001:
mode: fixed
layout:
collision_margin_m: 0.02
max_slsqp_iterations: 500
sampling:
seed: 42
candidate_count: 100
max_attempts: 1000
diversity:
enabled: true
count: 20
output:
directory: ./scene_layouts
The seed must be a nonnegative integer; counts and iteration limits must be positive integers. Geometric Constraints and Solver InputsLet c0 be the baseline group center, h its half-extents, and Smin/Smax the table's safe boundaries. If a displacement window is configured, intersect this interval with
Bounds describe the allowed region for the complete AABB, not its center interval. Use the shared Sampling
The shared solver uses the current random centers c_sample as its reference and minimizes weighted Diversity SelectionUse incremental max-min selection after generating the pool, comparing only the XY positions W/H are the table dimensions, and sampled groups have equal weight. Remove identical layouts first, Stop when M is reached, the pool is exhausted, or the maximum difference is 0. Layout Reconstruction and ApplicationWhen loading a layout, verify source/base/policy digests, complete group coverage, finite centers, For example, moving the tray AABB center from Currently, procedural.generate updates only entities listed in the layout; it does not automatically The host owns physical baseline restoration, B=1 /
Future ScopeRetain the earlier requirements: yaw defaults to 12 candidates evenly spaced over 360°, with |
Scene Engine follow-up: grouped scene randomizationPR #761 is merged as First-version scope: rigid-body support subtrees on one horizontal, connected table surface without holes; fixed orientation and Z height; XY translation using whole-group AABBs. Place export/graph/coordinate and file-format adaptation under 1. Freeze the data and interface contracts
Acceptance: one minimal fixture can be serialized, loaded and mapped to the shared contracts, with invalid inputs rejected explicitly. 2. Prepare and cache the common baseline
Acceptance: cover nested groups, offset root origins, nonuniform scale, coordinate conversion, cache hits/invalidation, unsupported surfaces and unchanged source files. 3. Generate and persist the candidate pool
Acceptance: deterministic replay for the same inputs/seed/version; correct fixed-group and displacement behavior; accurate budget exhaustion; portable references and detectable tampering/incomplete output. 4. Select diverse layouts
Acceptance: selection leaves layout files unchanged, is reproducible, and reports actual selected counts. This can follow the generation/application workflow. 5. Adapt layouts for physical consumption
Acceptance: a root → child → grandchild fixture moves together; an offset AABB center is handled correctly; switching A → B → A causes no accumulated displacement; a member left behind is rejected by measured validation. Integration ownership
Geometric Suggested delivery order: baseline/contracts → generation + layout I/O → full-member application and measured integration → diversity. Yaw, overhang, distractors, multiple support surfaces and batched physical mutation remain later capabilities. |
Description
GenSim generation/editing, procedural recipes and Task Program scene expansion now share
embodichain.lab.sim.sceneservices for configuration composition, placement, layout, physical state, deterministic operations and validation. A model-service-free tabletop recipe builds two containers, parts and a keep-out region, then prepares, settles, re-measures and restores the scene.The numerical boundary is deliberately small:
compute.geometryexports five pure Torch functions for AABB intersection, containment violation, masked group-height reduction, affine box transforms and triangle normals/areas. They support broadcast batch dimensions, float32/float64 and CPU/CUDA without reading tensor values on the host. Scene configuration, UID mappings, polygon/mesh preprocessing, logging, candidate search and SLSQP remain in scene-owned CPU modules. No general backend/solver framework is introduced. The earlier draft's compute-side authoring classes andcompute.layoutnamespace were removed; established scene and GenSim entry points remain compatible.This PR absorbs #749, retaining its compatibility imports, proposal identities, movable permissions and rejected-recording gates. Main's current dynamic Task Program language behavior is preserved. The concrete workspace/Gym host remains in #750 and consumes the shared services. Numerical batches do not imply batched physical mutation: the runtime integration remains B=1.
Dependencies: the standard EmbodiChain installation uses SciPy and trimesh supplied by DexSim and directly requires
shapely>=2.0for its placement API. No separatesceneextra or additional scene installation step is needed. The tensor kernels require Torch only. No new physics backend or model provider.Refs #760 (module roadmap), #748 (Task Program expansion workflow). Supersedes closed #749; later scene capabilities remain tracked in #760.
Stack
mainType of change
Validation
Final dependency/review fixes: 506 focused scene, GenSim, Task Program/demo and context tests passed. Wheel metadata verifies unconditional
shapely>=2.0and nosceneextra; Black, API coverage (2589/2589), context and whitespace gates passed.Rectangular projection now computes boundary offsets directly and rounds inward before exact containment validation; decimal-coordinate regressions and 150 boundary probes passed while oversized AABBs remain rejected.
Snapshot restoration rejects saved qpos/position targets outside current limits before any entity writes. Regression coverage includes excursions, limits changed after capture and a round trip through the production articulation setter. Native/GPU qualification below is from the earlier implementation; these final changes were validated with CPU tests.
Example/tutorial visualization coverage and API-doc tooling: 9 checks passed. The tabletop recipe explicitly disables browser visualization and rendering, and its native proposal/convergence/layout/restoration checks passed again on
9c78251f.The scene reference and Scene Engine guide now list the shared adapters, distinguish the explicit recipe path, and document the B=1 physical runtime versus batched tensor-kernel scope. Existing routed context remains accurate; no context change is needed for this example/documentation fix.
452 CPU/integration/context/docs checks passed; 37 CUDA cases were skipped in that CPU run and then all 37 passed in an explicit, serial CUDA run.
Coverage includes GenSim Scene Engine, scene proposals and physical services, padding/empty batches, masks, dtype/device preservation, transformed-box corner oracles, and numerical degeneracy without implicit host synchronization.
Independent review preserved contained/zero-sized AABB solver semantics and kept single-pair collision queries constant-sized instead of building an N-by-N tensor.
Benchmark reference checking exposed and fixed degenerate-triangle cancellation residuals being normalized into false unit normals; no tolerance was loosened.
Native B=1 CPU/no-render tabletop recipe passed proposal, convergence, measured XY/keep-out checks and state restoration with the existing proposal identity.
101 architecture/context/docs tests passed. Black 26.3.1
black ., whitespace and context gates passed; API coverage is 2589/2589.Standard fresh Sphinx dummy build passed with 830 existing warnings; new scene/compute/benchmark pages and architecture have no new warnings.
Geometry benchmark
Reproduce with
python -m scripts.benchmark.scene.geometry_kernels --device cuda:0 --iterations 100. The report includes CPU/GPU memory, an independent NumPy float64 accuracy reference and all measured implementations. Measurements below use float32, 4 Torch CPU threads and an RTX PRO 5000 72GB Blackwell on a shared machine; B is independent layouts/candidates and N is objects per layout.All accuracy checks passed (maximum absolute numerical error 3.83e-6 across outputs); largest measured CUDA peak was 67.35 MiB. Small CPU workloads remain appropriate for CPU execution. These timings cover the geometry workload, not the SLSQP solver or an entire scene pipeline.
Horizontal/fixed-orientation proposals and proxy checks remain explicit limits. Changed orientations require measured footprints. Full solver checkpoints, batched physical mutation, exact mesh collision and general room generation are not qualified in this phase.
Screenshots
No UI change.
Checklist
black .with Black 26.3.1.