MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation

This repository provides a PyTorch implementation of the paper MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation.

An extended version has been published in TPAMI Link.

Tested with:

• PyTorch 0.4.1
• Python 2.7.12

Qualitative Results:

Training:

• Download the data folder, which contains the features and the ground truth labels. (~30GB) (If you cannot download the data from the previous link, try to download it from here)
• Extract it so that you have the data folder in the same directory as main.py.
• To train the model run python main.py --action=train --dataset=DS --split=SP where DS is breakfast, 50salads or gtea, and SP is the split number (1-5) for 50salads and (1-4) for the other datasets.

Prediction:

Run python main.py --action=predict --dataset=DS --split=SP.

Evaluation:

Run python main.py --action=eval --split=SP

Citation:

If you use the code, please cite

Y. Abu Farha and J. Gall.
MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019

S. Li, Y. Abu Farha, Y. Liu, MM. Cheng,  and J. Gall.
MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation.
In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020

Real install instructions

conda create --name ptg python=3.10 scikit-learn jupyter seaborn pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install ubelt