Skip to main content

Jiawei Yan

Maintaining privacy in human behaviour analysis.

Email: j.yan11@ncl.ac.uk

Project title

Contextual information for action recognition and behaviour analysis

Supervisors

Project description

We are investigating methods of human behaviour analysis. We are targeting:

  • detection and localisation
  • contextual information
  • 2D pose features fusion
  • multimodal processing

Methodology and objectives

To recognise human action, there are two main aspects to consider, which are target data and contextual data. Target data refers to those frame pixels which describe the human body. In contrast, contextual data refers to every other pixel which describes the background and the visible objects.

We have implemented Mask-RCNN to provide semantic segmentation between background and human targets. This provides high-quality segmentation masks of detected objects.

We use Mask-RCNN to pre-process the data. We then inject frames into the state-of-the-art Inflated 3D ConvNet (I3D) network for human action recognition.

Result

We propose a method to preserve the target privacy for those applications where privacy-protection is an issue. We have based the method on an image segmentation mask to occlude the target-related data.

We have demonstrated that the resulting network performs human action recognition with similar performance when compared to the baseline. Our research has highlighted the importance played by contextual information. The information is crucial in providing high human action recognition performance.

Interests

Football.

Qualifications

  • MSc in Automation and Control from Newcastle University