Spark SQL 按主键 MERGE（湖表）
Spark SQL 按主键 MERGE（湖表）
MERGE INTO lake.orders AS t
USING cdc_stage AS s
ON t.order_id = s.order_id
WHEN MATCHED AND s.op = 'd' THEN DELETE
WHEN MATCHED THEN UPDATE SET t.amount = s.amount, t.status = s.status
WHEN NOT MATCHED THEN INSERT (order_id, amount, status)
VALUES (s.order_id, s.amount, s.status);| Hudi | COW/MOR 两种表类型，支持 upsert 与增量读 |
|---|---|
| Iceberg | MERGE INTO + 行级删除（v2 规范） |
| Delta | MERGE 与变更数据流 |
| 乱序 | 按事件时间做预聚合/取最新，避免旧事件覆盖新值 |
高频 merge 会产生小文件：需配套 compaction（Hudi clustering、Iceberg rewrite、Delta OPTIMIZE）。