Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2125 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1838. Min Act: -0.1147

Data Index: 38059 (Open Web Text)

Max Activating Token Index: 108

Text #1

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1567. Min Act: -0.1320

Data Index: 7805919 (Open Web Text)

Max Activating Token Index: 181

Text #2

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1323. Min Act: -0.1291

Data Index: 7952950 (Open Web Text)

Max Activating Token Index: 663

Text #3

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1322. Min Act: -0.1419

Data Index: 4508729 (Open Web Text)

Max Activating Token Index: 533

Text #4

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1372. Min Act: -0.1302

Data Index: 4130863 (Open Web Text)

Max Activating Token Index: 556

Text #5

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1231. Min Act: -0.1563

Data Index: 1853418 (Open Web Text)

Max Activating Token Index: 339

Text #6

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1300. Min Act: -0.1416

Data Index: 104490 (Open Web Text)

Max Activating Token Index: 573

Text #7

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1134. Min Act: -0.1566

Data Index: 8218514 (Open Web Text)

Max Activating Token Index: 580

Text #8

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1104. Min Act: -0.1524

Data Index: 298580 (Open Web Text)

Max Activating Token Index: 645

Text #9

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1142. Min Act: -0.1497

Data Index: 2569348 (Open Web Text)

Max Activating Token Index: 624

Text #10

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1130. Min Act: -0.1393

Data Index: 6649595 (Open Web Text)

Max Activating Token Index: 614

Text #11

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1111. Min Act: -0.1529

Data Index: 5408894 (Open Web Text)

Max Activating Token Index: 123

Text #12

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1129. Min Act: -0.1356

Data Index: 2405133 (Open Web Text)

Max Activating Token Index: 103

Text #13

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1124. Min Act: -0.1472

Data Index: 323386 (Open Web Text)

Max Activating Token Index: 129

Text #14

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1141. Min Act: -0.1440

Data Index: 3189512 (Open Web Text)

Max Activating Token Index: 683

Text #15

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1136. Min Act: -0.1558

Data Index: 5198772 (Open Web Text)

Max Activating Token Index: 89

Text #16

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1044. Min Act: -0.1563

Data Index: 7721778 (Open Web Text)

Max Activating Token Index: 95

Text #17

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.0977. Min Act: -0.1166

Data Index: 1036120 (Open Web Text)

Max Activating Token Index: 251

Text #18

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1031. Min Act: -0.1682

Data Index: 7937402 (Open Web Text)

Max Activating Token Index: 650

Text #19

Max Range: 1.1838. Min Range: -1.1838

Max Act: 1.1030. Min Act: -0.1670

Data Index: 2003425 (Open Web Text)

Max Activating Token Index: 80