Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1121 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.4527. Min Act: -0.1700

Data Index: 4672689 (Open Web Text)

Max Activating Token Index: 441

Text #1

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.3375. Min Act: -0.1700

Data Index: 8235454 (Open Web Text)

Max Activating Token Index: 14

Text #2

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2849. Min Act: -0.1700

Data Index: 7072976 (Open Web Text)

Max Activating Token Index: 123

Text #3

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2684. Min Act: -0.1700

Data Index: 5756816 (Open Web Text)

Max Activating Token Index: 31

Text #4

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2587. Min Act: -0.1700

Data Index: 8695894 (Open Web Text)

Max Activating Token Index: 369

Text #5

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2492. Min Act: -0.1700

Data Index: 5270808 (Open Web Text)

Max Activating Token Index: 219

Text #6

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2487. Min Act: -0.1700

Data Index: 2363935 (Open Web Text)

Max Activating Token Index: 433

Text #7

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2451. Min Act: -0.1700

Data Index: 2363934 (Open Web Text)

Max Activating Token Index: 88

Text #8

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2384. Min Act: -0.1700

Data Index: 8235453 (Open Web Text)

Max Activating Token Index: 592

Text #9

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2380. Min Act: -0.1700

Data Index: 2605420 (Open Web Text)

Max Activating Token Index: 385

Text #10

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2224. Min Act: -0.1700

Data Index: 4567824 (Open Web Text)

Max Activating Token Index: 646

Text #11

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2432. Min Act: -0.1700

Data Index: 2548292 (Open Web Text)

Max Activating Token Index: 349

Text #12

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2395. Min Act: -0.1700

Data Index: 6797791 (Open Web Text)

Max Activating Token Index: 20

Text #13

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.2270. Min Act: -0.1700

Data Index: 2334029 (Open Web Text)

Max Activating Token Index: 91

Text #14

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.1729. Min Act: -0.1700

Data Index: 999627 (Open Web Text)

Max Activating Token Index: 241

Text #15

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.1847. Min Act: -0.1700

Data Index: 1876364 (Open Web Text)

Max Activating Token Index: 154

Text #16

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.1891. Min Act: -0.1700

Data Index: 613359 (Open Web Text)

Max Activating Token Index: 37

Text #17

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.1757. Min Act: -0.1700

Data Index: 7271776 (Open Web Text)

Max Activating Token Index: 23

Text #18

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.1570. Min Act: -0.1700

Data Index: 4311791 (Open Web Text)

Max Activating Token Index: 1014

Text #19

Max Range: 4.4527. Min Range: -4.4527

Max Act: 4.1426. Min Act: -0.1700

Data Index: 490024 (Open Web Text)

Max Activating Token Index: 24