Andrii Boichuk: Video-based action recognition on edge AI & BigData Online Day 2023 Spring Website - www.aiconf.com.ua Youtube - https://www.youtube.com/startuplviv FB - https://www.facebook.com/aiconf

1. VIDEO-BASED ACTION
RECOGNITION ON EDGE
01.04.2023

2. ● How often a senior is treated in emergency room due to a fall?
2

3. ● How often a senior is treated in emergency room due to a fall? - every 13 minutes
3

4. ● How often a senior is treated in emergency room due to a fall? - every 13 minutes
● How much money do people spend due to falls?
4

5. ● How often a senior is treated in emergency room due to a fall? - every 13 minutes
● How much money do people spend due to falls? - 69 billion $ annually
5

6. 6
Structure
What
1
Why
2
How
3
Q&A
4

7. 7
Who am I?
ANDRII
BOICHUK
Research Engineer
● Lecturer at NU “Lviv Polytechnic”
● 10+ year in IT, 5+ in research
● Tons of research activity

8. 8
Who am I? (SQUAD Edition)
FOLLOW ON
FACEBOOK
FOLLOW ON
LINKEDIN
FOLLOW ON
INSTAGRAM
1250 +
Research Delivery
QA Embedded Hardware Mobile
Infrastructure Product Design
Kyiv
Lviv
Uzhhorod
STREAMS OF
TECHNOLOGIES
SQUAD – one of the biggest RnD center
in Ukraine. SQUAD – has some of the best
optical labs in Europe.

9. ● 500+ студентів
● Вступний бал 190+
● 50% викладачів з галузі
9
Who am I? (NULP edition)
https://aidept.com.ua/

10. What?

11. ● Action recognition
11
What is video-based action recognition on edge?

12. 12
Action recognition
Action classification seeks to assign the correct label (e.g. “cooking,” “writing,” etc.)
to a given image or video.
Action localization, given a particular action and a video as input, seeks to identify
the correct location and timestamp in the video when the action is being performed.

13. ● Action recognition
● Video based
13
What is video-based action recognition on edge?

14. ● Action recognition
● Video based
14
What is video-based action recognition on edge?

15. ● Action recognition
● Video based
● On edge
15
What is video-based action recognition on edge?

16. 16
What is video-based action recognition on edge?
● Action recognition
● Video based
● On edge

17. Why?

18. 18
Why action recognition?
● Saves lives

19. 19
Saves lives

20. 20
Why action recognition?
● Saves lives
● Saves lives and money

21. 21
Saves lives and money

22. 22
Why action recognition?
● Saves lives
● Saves lives and money
● Gives peace of mind

23. 23
Gives peace of mind Most of the times

24. 24
Why action recognition?
● Saves lives
● Saves lives and money
● Gives peace of mind
● Saves time

25. 25
Saves time - sport highlights

26. 26
Cloud problems
● Privacy

27. 27
Cloud problems
● Privacy
● “bad internet connection”

28. 28
Cloud problems
● Privacy
● “bad internet connection”
● Expensive

29. 29
Cloud problems
● Privacy
● “bad internet connection”
● Expensive
● Streaming latency

30. 30
Solution?
● Edge!

31. Why not?

32. ● Harder
32

33. ● Harder - HW, optimizations techniques and
combinations
33

34. ● Harder
● Worse accuracy
34

35. ● Harder
● Worse accuracy
● Uses additional resources which might be needed for
other tasks
35

36. Where?

37. 37
Where?
● Surveillance cameras

38. 38
Where

39. 39
Where?
● Smart home cameras
● Fall detection

40. 40
Where

41. How?

42. 42
How
● Research

43. 43
How
● Research - always check for SoTA - paperswithcode, github, CVPR,
EECV

44. 44
How
● Research
● Hardware

45. 45
How
● Research
● Hardware - know your limitations - CPU, GPU, FPGA or SoC?

46. 46
How
● Research
● Hardware
● Optimizations

47. 47
How
● Research
● Hardware
● Optimizations - quantization, pruning and hardware independent
techniques

48. 48
How
● Research
● Hardware
● Optimizations

49. - CNN-based model
- GPU as hardware
- Pros: high throughput, high performance, no
optimizations needed
- Cons: memory latency, high energy consumption,
might be expensive
49
How

50. - Transformer-based model
- CPU as hardware
- Pros: low latency, universal, easy to use and optimize
- Cons: mid performance on other stats - mid energy
consumption, mid speed
50
How

51. 51
How
● Research
● Hardware
● Optimizations
● Make your problem as narrow as possible

52. - Have the most basic baseline ready ASAP
- Chose target HW as soon as possible
- Go for SoTA
52
Tips

53. Q&A

54. Andrii Boichuk
Research engineer
Thank You!
SQUAD IN
FACEBOOK
SQUAD IN
LINKEDIN
SQUAD IN
INSTAGRAM