One of the greatest goals of AI is building an artificial continuous learning agent which can construct a sophisticated understanding about the external world from its own experience through the adaptive, goal-oriented and incremental development of ever more complex skills and knowledge. Yet, Continuous/Lifelong Learning (CL) from high-dimensional streaming data is a challenging research problem far from being solved. In fact, fully retraining deep prediction models each time a new piece of data becomes available is infeasible, due to computational and storage issues, while naïve continuous learning strategies have been shown to suffer from catastrophic forgetting. This talk will cover some of the most common end-to-end continuous learning strategies for gradient-based architectures and the recently proposed AR-1 strategy, which can outperform other state-of-the-art regularization and architectural approaches on the CORe50 benchmark.
7. • Higher and realistic time-scale where data (and tasks)
become available only during time
• No access to previously encountered data.
• Constant computational and memory resources.
• Incremental development of ever more complex
knowledge and skills.
8.
9. Dataset, Benchmark, code and additional
information freely available at:
vlomonaco.github.io/core50
LomonacoV. and Maltoni D. CORe50: a New Dataset and Benchmark for ContinuousObject Recognition.CoRL2017.
19. Combining Architectural and Regularization
approaches
LomonacoV. and Maltoni D. Continuous Learning in Single-Incremental-TaskScenarios.To be published.
24. 𝑊𝑡𝑊𝑏...
𝐿0, … , 𝐿 𝑛−1 𝐿 𝑛
• Computational efficient
(independent from the
number of training
batches)
• Just one Ω 𝑘(running sum
+ max clip)
• CWR with zero-init
• CWR with mean-shift
AR-1 (Aerojet Rocketdyne) name of the booster under development in competition with the blue origin BE-4 to replace Russuan RD-180. Falcon 9 has booster B1029. united lunch alliance atals v