New Neural Trick Helps Models Think in Longer Patterns

1 Apr 2025

Authors:

(1) Hung Le, Applied AI Institute, Deakin University, Geelong, Australia;

(2) Dung Nguyen, Applied AI Institute, Deakin University, Geelong, Australia;

(3) Kien Do, Applied AI Institute, Deakin University, Geelong, Australia;

(4) Svetha Venkatesh, Applied AI Institute, Deakin University, Geelong, Australia;

(5) Truyen Tran, Applied AI Institute, Deakin University, Geelong, Australia.

Table of Links

Abstract & Introduction

Methods

Methods Part 2

Experimental Results

Experimental Results Part 2

Related Works, Discussion, & References

Appendix A, B, & C

Appendix D

2.3 Pointer-Augmented Neural Memory (PANM)

2.3.1 Pointer Unit

2.3.2 Pointer-based Addressing Modes

2.3.3 The Controller

Table 1: Algorithmic reasoning: mean sequence-level accuracy (%) over testing lengths Other Max is selected as the best numbers at each length mode from other baselines.

Table 2: SCAN (Left): Exact match accuracy (%, median of 5 runs) on splits of various lengths. Mathematics (Right): mean accuracy over 5 runs. The baselines’ numbers are from Csord´as et al. [2021] and we run PANM using the authors’ codebase.

This paper is available on arxiv under CC BY 4.0 DEED license.

← Previous

A New Neural Memory Trick Helps AI Handle Much Longer Sequences

Up Next →

A Memory Hack That Makes AI Smarter With Sequences