ThunderVVV commited on
Commit
011fbcc
·
1 Parent(s): 5f028d6
Files changed (1) hide show
  1. README.md +13 -94
README.md CHANGED
@@ -1,94 +1,13 @@
1
- <div align="center">
2
-
3
- # HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos
4
-
5
- [Jinglei Zhang]()<sup>1</sup> &emsp; [Jiankang Deng](https://jiankangdeng.github.io/)<sup>2</sup> &emsp; [Chao Ma](https://scholar.google.com/citations?user=syoPhv8AAAAJ&hl=en)<sup>1</sup> &emsp; [Rolandos Alexandros Potamias](https://rolpotamias.github.io)<sup>2</sup> &emsp;
6
-
7
- <sup>1</sup>Shanghai Jiao Tong University, China
8
- <sup>2</sup>Imperial College London, UK <br>
9
-
10
- <a href='https://hawor-project.github.io/'><img src='https://img.shields.io/badge/Project-Page-blue'></a>
11
- <a href='https://arxiv.org/abs/'><img src='https://img.shields.io/badge/Paper-arXiv-red'></a>
12
- </div>
13
-
14
- This is the official implementation of **[HaWoR](https://hawor-project.github.io/)**, a hand reconstruction model in the world coordinates:
15
-
16
- ![teaser](assets/teaser.png)
17
-
18
- ## Installation
19
-
20
- ### Installation
21
- ```
22
- git clone --recursive https://github.com/ThunderVVV/HaWoR.git
23
- cd HaWoR
24
- ```
25
-
26
- The code has been tested with PyTorch 1.13 and CUDA 11.7. It is suggested to use an anaconda environment to install the the required dependencies:
27
- ```bash
28
- conda create --name hawor python=3.10
29
- conda activate hawor
30
-
31
- pip install torch==1.13.0+cu117 torchvision==0.14.0+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
32
- # Install requirements
33
- pip install -r requirements.txt
34
- pip install pytorch-lightning==2.2.4 --no-deps
35
- pip install lightning-utilities torchmetrics==1.4.0
36
- ```
37
-
38
- ### Install masked DROID-SLAM:
39
-
40
- ```
41
- cd thirdparty/DROID-SLAM
42
- python setup.py install
43
- ```
44
-
45
- Download DROID-SLAM official weights [droid.pth](https://drive.google.com/file/d/1PpqVt1H4maBa_GbPJp4NwxRsd9jk-elh/view?usp=sharing), put it under `./weights/external/`.
46
-
47
- ### Install Metric3D
48
-
49
- Download Metric3D official weights [metric_depth_vit_large_800k.pth](https://drive.google.com/file/d/1eT2gG-kwsVzNy5nJrbm4KC-9DbNKyLnr/view?usp=drive_link), put it under `thirdparty/Metric3D/weights`.
50
-
51
- ### Download the model weights
52
-
53
- ```bash
54
- wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt -P ./weights/external/
55
- wget https://huggingface.co/ThunderVVV/HaWoR/resolve/main/hawor/checkpoints/hawor.ckpt -P ./weights/hawor/checkpoints/
56
- wget https://huggingface.co/ThunderVVV/HaWoR/resolve/main/hawor/checkpoints/infiller.pt -P ./weights/hawor/checkpoints/
57
- wget https://huggingface.co/ThunderVVV/HaWoR/resolve/main/hawor/model_config.yaml -P ./weights/hawor/
58
- ```
59
- It is also required to download MANO model from [MANO website](https://mano.is.tue.mpg.de).
60
- Create an account by clicking Sign Up and download the models (mano_v*_*.zip). Unzip and put the hand model to the `_DATA/data/mano/MANO_RIGHT.pkl` and `_DATA/data_left/mano_left/MANO_LEFT.pkl`.
61
-
62
- Note that MANO model falls under the [MANO license](https://mano.is.tue.mpg.de/license.html).
63
- ## Demo
64
-
65
- For visualizaiton in world view, run with:
66
- ```bash
67
- python demo.py --video_path ./example/video_0.mp4 --vis_mode world
68
- ```
69
-
70
- For visualizaiton in camera view, run with:
71
- ```bash
72
- python demo.py --video_path ./example/video_0.mp4 --vis_mode cam
73
- ```
74
-
75
- ## Training
76
- The training code will be released soon.
77
-
78
- ## Acknowledgements
79
- Parts of the code are taken or adapted from the following repos:
80
- - [HaMeR](https://github.com/geopavlakos/hamer/)
81
- - [WiLoR](https://github.com/rolpotamias/WiLoR)
82
- - [SLAHMR](https://github.com/vye16/slahmr)
83
- - [TRAM](https://github.com/yufu-wang/tram)
84
- - [CMIB](https://github.com/jihoonerd/Conditional-Motion-In-Betweening)
85
-
86
-
87
- ## License
88
- HaWoR models fall under the [CC-BY-NC--ND License](./license.txt). This repository depends also on [MANO Model](https://mano.is.tue.mpg.de/license.html), which are fall under their own licenses. By using this repository, you must also comply with the terms of these external licenses.
89
- ## Citing
90
- If you find HaWoR useful for your research, please consider citing our paper:
91
-
92
- ```bibtex
93
-
94
- ```
 
1
+ ---
2
+ title: HaWoR
3
+ emoji: 🚀
4
+ colorFrom: indigo
5
+ colorTo: red
6
+ sdk: gradio
7
+ sdk_version: 4.44.0
8
+ python_version: 3.10
9
+ app_file: app.py
10
+ pinned: true
11
+ license: cc-by-nc-2.0
12
+ ---
13
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference