Task_160 · Video Object Coordinate Map Estimation

这个 task 到底在学什么？

一句话：模型要判断视频中每个可见物体表面点，在该物体自己的 normalized 3D coordinate system 里处于哪个 XYZ 位置。

1 · Input 看外观普通 RGB video 只告诉模型像素长什么样、物体如何运动。

2 · Target 给逐像素 XYZTarget 的 R/G/B 三个 channel 分别编码 normalized X/Y/Z；黑色是背景。

3 · 跨帧保持几何对应同一个物理表面点即使随物体或相机移动，也应维持近似相同的 coordinate value。

Depth	问“这个点离相机多远”。
Segmentation	问“这个像素属于哪个物体/类别”。
Object coordinate map	问“这个像素在该物体自身 normalized XYZ 中位于哪里”；可支持 dense correspondence、pose、tracking、size 和 reconstruction。

注意：不同物体上的相同颜色只表示它们各自在自身 bounding box 中的位置相似，并不表示相同类别或语义部件。

How to read the videos

左边是 Input RGB；右边是由 source-native uint16 coordinates 与 segmentation 确定性构造的 Target display proxy。右边不是模型 prediction。

Artifact boundary

这是独立 pre-pipeline Demo，不是 Raw Dataset Report，也不改变 full-production census。Open source-native Dataset Report

Metadata 状态：团队当前规范是 Metadata Contract 1.2（读取自 dev@4ec8eb60e13692d1b7dfb59d0f3ba4f2e0a99402）。本页现在直接展开每份 JSON。Task 160 尚无 committed pipeline，因此 pipeline.commit 只能保持明确的 pending placeholder；这些 records 是可审阅的 1.2 layout drafts，不是假装通过正式 validator 的 Metadata。

SAMPLE 01

test:218

DEMO VERIFIED

较稀疏、多物体长条结构的 scene；适合检查 background masking 与细长物体表面的 coordinate continuity。

INPUT · RGB VIDEO

Source-native RGB frames；presentation/model-ingestion H.264 transcode。

TARGET · OBJECT COORDINATE MAP

黑背景 8-bit NOCS display proxy；numeric supervision 使用 lossless RGB Target。

Prompt

Estimate the normalized object-coordinate map for every visible object surface in each frame; render background as black.

Sourcemovi_e-test.tfrecord-00230-of-00256 · record 0 · frames 0–23

Input181,439 bytes · SHA-256 063e09e6…d07b84

Lossless Target367,964 bytes · gbrp · exact-pixel readback PASS

Target derivationround(uint16/257) + segmentation > 0 foreground mask

直接查看 Metadata 1.2 draft

PRE-PIPELINE DRAFT — 暂不通过正式 validator。 字段结构、顺序、sample ID 和 fingerprint 已按 Metadata 1.2；唯一尚未具备的正式 provenance 是 pipeline.commit，因为 Task 160 pipeline 还没有进入 tv2v-reasoning 的 committed clean worktree。

{
  "schema_version": "1.2",
  "sample_id": "Task_160_video_object_coordinate_map_estimation_000000001",
  "task": {
    "id": "Task_160",
    "name": "video_object_coordinate_map_estimation",
    "modality": "v2v"
  },
  "source": {
    "dataset": "MOVi-E_TFDS_1.0.0_256x256",
    "split": "test",
    "sample_id": "test:218",
    "uri": "s3://tv2v-data/raw/taxonomy/Task_160_video_object_coordinate_map_estimation/movi_e/256x256/1.0.0/movi_e-test.tfrecord-00230-of-00256"
  },
  "media": {
    "input": {
      "file": "input_video.mp4",
      "width": 256,
      "height": 256,
      "fps": 12.0,
      "num_frames": 24
    },
    "target": {
      "file": "target_video.mp4",
      "width": 256,
      "height": 256,
      "fps": 12.0,
      "num_frames": 24
    }
  },
  "pipeline": {
    "name": "video_object_coordinate_map_estimation",
    "version": "0.0.0-demo",
    "commit": "<PENDING_FORMAL_PRODUCING_COMMIT>"
  },
  "optional": {
    "source_shard": "movi_e-test.tfrecord-00230-of-00256",
    "source_record_index": 0,
    "source_frame_start": 0,
    "source_frame_end": 23
  },
  "identity": {
    "sample_fingerprint": "d34b813d88750b6649df696e7672af9e6fcb5495988cfaee9981a6088e960059"
  }
}

查看 Demo derivation / verification provenance

{
  "artifact_state": "DEMO VERIFIED; not Dataset Final",
  "display": {
    "bytes": 75039,
    "encoding": "H.264 yuv420p CRF 15, 12 FPS",
    "probe": {
      "codec": "h264",
      "fps": "12/1",
      "frames": 24,
      "height": 256,
      "pixel_format": "yuv420p",
      "width": 256
    },
    "sha256": "0da822dd8ffa5ca3453e78322a702d8ccca638998fec523f7668292c1589691c",
    "status": "presentation-only lossy transcode; never use as numeric supervision",
    "target_proxy": "display/Task_160_video_object_coordinate_map_estimation_000000001/target_display_proxy.mp4"
  },
  "input": {
    "bytes": 181439,
    "file": "input_video.mp4",
    "probe": {
      "codec": "h264",
      "fps": "12/1",
      "frames": 24,
      "height": 256,
      "pixel_format": "yuv420p",
      "width": 256
    },
    "sha256": "063e09e68761ed7e201ffd49746406d26df1d6ac94da0fd633336cccf4d07b84",
    "status": "source-native RGB frames; deterministic display/model-ingestion transcode",
    "transform": "24 native RGB PNG frames encoded at 12 FPS with H.264 yuv420p CRF 15; no crop, resize, trim, interpolation, or audio"
  },
  "overlap": {
    "cross_sample": "0; distinct complete source examples",
    "within_sample": "Input and Target use the same 24 source timestamps"
  },
  "prompt": {
    "file": "prompt.txt",
    "sha256": "657fcdfe5acbd7017dae3ddd9362f70d612b6c1dd3ee923b0dfc96015d046c97",
    "text": "Estimate the normalized object-coordinate map for every visible object surface in each frame; render background as black."
  },
  "sample_id": "test_218",
  "schema_version": "task160-demo-provenance-1.1",
  "source": {
    "dataset": "MOVi-E TFDS 1.0.0 256x256",
    "fps": 12,
    "frame_bounds": [
      0,
      23
    ],
    "frames": 24,
    "native_object_coordinates_payload_sha256": "a56a5a804fe7479a29f6a471df47fe1d608260440e026b03055b2ae7c7956414",
    "native_rgb_payload_sha256": "60d0d177a22618e583cdec63e935f567cafd6d49c1cdf5e016e956c9dbaf7fda",
    "native_segmentations_payload_sha256": "77230c158922cf2da6b23d4c8576795d61fe51062c1ae84ed4ba7c11282b71fd",
    "pairing_key": "test:218",
    "record_index": 0,
    "resolution": [
      256,
      256
    ],
    "role": "Source A",
    "shard": "movi_e-test.tfrecord-00230-of-00256",
    "split": "test",
    "video_name": "218"
  },
  "supervision_class": "B",
  "target": {
    "background_pixels": 1305184,
    "bytes": 367964,
    "encoding": "H.264 RGB CRF 0, gbrp, 12 FPS; exact 24-frame pixel readback PASS",
    "file": "target_video.mp4",
    "foreground_pixels": 267680,
    "probe": {
      "codec": "h264",
      "fps": "12/1",
      "frames": 24,
      "height": 256,
      "pixel_format": "gbrp",
      "width": 256
    },
    "readback_max_channel_error": 0,
    "sha256": "bb27213b944cf02ba24c04d70d682fa44684839f2486f06e19683be13ac5b3c7",
    "status": "deterministically annotation-derived from source-native uint16 object coordinates and segmentation",
    "transform": "round(uint16_coordinate / 257) to uint8; keep values where segmentation > 0; set segmentation == 0 to RGB (0,0,0)"
  },
  "task_id": 160,
  "task_name": "Video Object Coordinate Map Estimation",
  "verification": {
    "formal_metadata_status": "BLOCKED until a committed Task 160 pipeline produces the sample",
    "input_probe_pass": true,
    "target_exact_pixel_readback_pass": true,
    "target_full_decode_pass": true,
    "target_probe_pass": true
  }
}

Download Input Download lossless Target Prompt View Metadata inline

SAMPLE 02

test:823