Real-World Challenge
Overview
The top eight participants in the Simulation Challenge will advance to the Real-World Challenge. In the Real-World Challenge, participants are required to deploy their models and conduct real-robot evaluations on site at ICRA 2026, performing garment folding tasks on clothes with different deformation states. The categories of garments used in the real-world evaluation are the same as those in the simulation challenge (i.e. long-sleeved tops, short-sleeved tops, long pants, and shorts).
Features & Deformations
The Real-World Challenge primarily focuses on evaluating a model's generalization capability when faced with different garment deformations. Therefore, for each garment, we define three distinct deformation configurations, namely:
Important Notes
For each garment category (4 categories total), 5 garment items (including 3 seen garments and 2 unseen garments) will be tested under each of the three initial states. Leading to 15 experimental trials per category and 60 trials per team. Each trial is subject to a time limit. The initial states of all the testing garments will be set by on-site referees to ensure fairness and impartiality in the testing of each sample.
For teams adopting Imitation Learning in the physical phase, an official folding dataset is provided. This dataset includes 5 garment (don't involve 2 unseen garment used in evaluation phase), each with three distinct initial states: 10 trajectories for the Easy state, 20 trajectories for the Middle state, and 20 trajectories for the Hard state, resulting in a total of 720 demonstration trajectories.
| Garment Category | Garment Style | #Garments (Physical) | Difficulty Level | Dataset Number | Total |
|---|---|---|---|---|---|
| Tops | Long Sleeve | 5 | Easy | 10 | 250 |
| Middle | 20 | ||||
| Hard | 20 | ||||
| Short Sleeve | 5 | Easy | 10 | ||
| Middle | 20 | ||||
| Hard | 20 | ||||
| Pants | Long | 5 | Easy | 10 | 250 |
| Middle | 20 | ||||
| Hard | 20 | ||||
| Short | 5 | Easy | 10 | ||
| Middle | 20 | ||||
| Hard | 20 |
In the real-world challenge, each procedure is divided into several sub-tasks, with scores assigned to each step. The detailed scoring criteria are presented in the following table.
| Difficulty Level | Garment Category | Step 1 (Scores) | Step 2 (Scores) | Step 3 (Scores) | Step 4 (Scores) | Step 5 (Scores) | Total Scores | If Unseen |
|---|---|---|---|---|---|---|---|---|
| Easy | Tops | Fold Sleeves Right (5) | Fold Sleeves Left (5) | Fold Up (5) | Fold Right (5) | - | 20 | x 1.5 |
| Long Pants | Fold Up (5) | Fold Left (5) | Fold Left (5) | - | - | 15 | x 1.5 | |
| Short Pants | Fold Up (5) | Fold Left (5) | - | - | - | 10 | x 1.5 | |
| Middle | Tops | Unfold (6) | Fold Sleeves Right (6) | Fold Sleeves Left (6) | Fold Up (6) | Fold Right (6) | 30 | x 1.5 |
| Long Pants | Unfold (6) | Fold Up (6) | Fold Left (6) | Fold Left (6) | - | 24 | x 1.5 | |
| Short Pants | Unfold (6) | Fold Up (6) | Fold Left (6) | - | - | 18 | x 1.5 | |
| Hard | Tops | Unfold (7) | Fold Sleeves Right (7) | Fold Sleeves Left (7) | Fold Up (7) | Fold Right (7) | 35 | x 1.5 |
| Long Pants | Unfold (7) | Fold Up (7) | Fold Left (7) | Fold Left (7) | - | 28 | x 1.5 | |
| Short Pants | Unfold (7) | Fold Up (7) | Fold Left (7) | - | - | 21 | x 1.5 |
Final rankings will be determined based on the total score accumulated from all trials. Cash prizes will be awarded to the top three teams:
