{"id":4672,"date":"2019-11-25T21:10:21","date_gmt":"2019-11-25T21:10:21","guid":{"rendered":"https:\/\/www.dinu.at\/profile\/home\/?p=4672"},"modified":"2019-11-25T21:10:31","modified_gmt":"2019-11-25T21:10:31","slug":"overcoming-catastrophic-forgetting-with-context-dependent-activations-xda-and-synaptic-stabilization","status":"publish","type":"post","link":"https:\/\/www.dinu.at\/profile\/home\/overcoming-catastrophic-forgetting-with-context-dependent-activations-xda-and-synaptic-stabilization\/","title":{"rendered":"Overcoming Catastrophic Forgetting with Context-Dependent Activations (XdA) and Synaptic Stabilization"},"content":{"rendered":"
\n\n \n\t\t\n\t\t
\n\t\t\t\t\t\t
\n
\n
\n \n\n \n
\n \n Abstract<\/strong>\nOvercoming Catastrophic Forgetting in neural networks is crucial to solving continuous learning problems. Deep Reinforcement Learning uses neural networks to make predictions of actions according to the current state space of an environment. In a dynamic environment, robust and adaptive life-long learning algorithms mark the cornerstone of their success. In this thesis we will examine an elaborate subset of algorithms countering catastrophic forgetting in neural networks and reflect on their weaknesses and strengths. Furthermore, we present an enhanced alternative to promising synaptic stabilization methods, such as Elastic Weight Consolidation or Synaptic Intelligence. Our method uses context-based information to switch between different pathways throughout the neural network, reducing destructive activation interference during the forward pass and destructive weight updates during the backward pass. We call this method Context-Dependent Activations (XdA). We show that XdA enhanced methods outperform basic synaptic stabilization methods and are a better choice for long task sequences. <\/div>\n \n\n <\/div>\n \t\t<\/div>\n\t\t <\/div>\n \n <\/div>\n \n\t\t\n\t\t\n\t\t
\n\t\t\t\t\t\t
\n
\n
\n \n\n \n
\n \n Thesis link<\/a>\n
\nGitHub link<\/a>\n <\/div>\n \n\n <\/div>\n \t\t<\/div>\n\t\t <\/div>\n \n <\/div>\n \n\t\t<\/div>\n","protected":false},"excerpt":{"rendered":"

Abstract Overcoming Catastrophic Forgetting in neural networks is crucial to solving continuous learning problems. Deep Reinforcement Learning uses neural networks to make predictions of actions according to the current state space of an environment. In a dynamic environment, robust and adaptive life-long learning algorithms mark the cornerstone of their success. In this thesis we will […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[1],"tags":[],"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7SrVj-1dm","jetpack-related-posts":[{"id":5142,"url":"https:\/\/www.dinu.at\/profile\/home\/a-neuro-symbolic-perspective-on-large-language-models-llms\/","url_meta":{"origin":4672,"position":0},"title":"A Neuro-Symbolic Perspective on Large Language Models (LLMs)","date":"22. January 2023","format":false,"excerpt":"We are excited to present our work, combining the power of a symbolic approach and Large Language Models (LLMs). Our Symbolic API bridges the gap between classical programming (Software 1.0) and differentiable programming (Software 2.0). Conceptually, our framework uses neural networks - specifically LLMs - at its core, and composes\u2026","rel":"","context":"In "General"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.dinu.at\/wp-content\/uploads\/2023\/01\/symai_logo.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":5080,"url":"https:\/\/www.dinu.at\/profile\/home\/ensemble-learning-for-domain-adaptation-by-importance-weighted-least-squares\/","url_meta":{"origin":4672,"position":1},"title":"Addressing Parameter Choice Issues in Unsupervised Domain Adaptation by Aggregation","date":"13. August 2022","format":false,"excerpt":"Abstract We study the problem of choosing algorithm hyper-parameters in unsupervised domain adaptation, i.e., with labeled data in a source domain and unlabeled data in a target domain, drawn from a different input distribution. We follow the strategy to compute several models using different hyper-parameters, and, to subsequently compute a\u2026","rel":"","context":"In "General"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5076,"url":"https:\/\/www.dinu.at\/profile\/home\/a-dataset-perspective-on-offline-reinforcement-learning\/","url_meta":{"origin":4672,"position":2},"title":"A Dataset Perspective on Offline Reinforcement Learning","date":"1. August 2022","format":false,"excerpt":"Abstract The application of Reinforcement Learning (RL) in real world environments can be expensive or risky due to sub-optimal policies during training. In Offline RL, this problem is avoided since interactions with an environment are prohibited. Policies are learned from a given dataset, which solely determines their performance. Despite this\u2026","rel":"","context":"In "General"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4528,"url":"https:\/\/www.dinu.at\/profile\/home\/internship-report\/","url_meta":{"origin":4672,"position":3},"title":"Internship Report","date":"31. August 2016","format":false,"excerpt":"Developing a Xamarin App for Handwritten Character Recognition using a Neural Network Company: Siemens Corporation Corporate Technology Institute: University of Applied Sciences Upper-Austria Field of Study: Software Engineering Author: Dinu Marius-Constantin \u00a0Prelude This internship report gives an overview how about my experiences\u00a0at Siemens Corporation Corporate Technology with the development of\u2026","rel":"","context":"In "Education"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5068,"url":"https:\/\/www.dinu.at\/profile\/home\/align-rudder-learning-from-few-demonstrations-by-reward-redistribution-2\/","url_meta":{"origin":4672,"position":4},"title":"Align-RUDDER: Learning From Few Demonstrations by Reward Redistribution","date":"10. March 2022","format":false,"excerpt":"Abstract Reinforcement learning algorithms require many samples when solving complex hierarchical tasks with sparse and delayed rewards. For such complex tasks, the recently proposed RUDDER uses reward redistribution to leverage steps in the Q-function that are associated with accomplishing sub-tasks. However, often only few episodes with high rewards are available\u2026","rel":"","context":"In "General"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":4697,"url":"https:\/\/www.dinu.at\/profile\/home\/align-rudder-learning-from-few-demonstrations-by-reward-redistribution\/","url_meta":{"origin":4672,"position":5},"title":"Align-RUDDER: Learning From Few Demonstrations by Reward Redistribution","date":"30. September 2020","format":false,"excerpt":"Reinforcement Learning algorithms require a large number of samples to solve complex tasks with sparse and delayed rewards. Complex tasks can often be hierarchically decomposed into sub-tasks. A step in the Q-function can be associated with solving a sub-task, where the expectation of the return increases. RUDDER has been introduced\u2026","rel":"","context":"In "General"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"builder_content":"Abstract<\/strong> Overcoming Catastrophic Forgetting in neural networks is crucial to solving continuous learning problems. Deep Reinforcement Learning uses neural networks to make predictions of actions according to the current state space of an environment. In a dynamic environment, robust and adaptive life-long learning algorithms mark the cornerstone of their success. In this thesis we will examine an elaborate subset of algorithms countering catastrophic forgetting in neural networks and reflect on their weaknesses and strengths. Furthermore, we present an enhanced alternative to promising synaptic stabilization methods, such as Elastic Weight Consolidation or Synaptic Intelligence. Our method uses context-based information to switch between different pathways throughout the neural network, reducing destructive activation interference during the forward pass and destructive weight updates during the backward pass. We call this method Context-Dependent Activations (XdA). We show that XdA enhanced methods outperform basic synaptic stabilization methods and are a better choice for long task sequences. \n Thesis link<\/a>
GitHub link<\/a>","_links":{"self":[{"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/posts\/4672"}],"collection":[{"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/comments?post=4672"}],"version-history":[{"count":14,"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/posts\/4672\/revisions"}],"predecessor-version":[{"id":4688,"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/posts\/4672\/revisions\/4688"}],"wp:attachment":[{"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/media?parent=4672"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/categories?post=4672"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dinu.at\/profile\/home\/wp-json\/wp\/v2\/tags?post=4672"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}