Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1101 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.1851. Min Act: -0.1700

Data Index: 1207269 (Open Web Text)

Max Activating Token Index: 416

Text #1

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.1579. Min Act: -0.1700

Data Index: 2994574 (Open Web Text)

Max Activating Token Index: 692

Text #2

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.1285. Min Act: -0.1700

Data Index: 2505882 (Open Web Text)

Max Activating Token Index: 388

Text #3

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.1080. Min Act: -0.1700

Data Index: 3360187 (Open Web Text)

Max Activating Token Index: 280

Text #4

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0968. Min Act: -0.1700

Data Index: 8123406 (Open Web Text)

Max Activating Token Index: 521

Text #5

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0857. Min Act: -0.1700

Data Index: 6060224 (Open Web Text)

Max Activating Token Index: 452

Text #6

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0829. Min Act: -0.1700

Data Index: 5469367 (Open Web Text)

Max Activating Token Index: 381

Text #7

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0684. Min Act: -0.1700

Data Index: 8209379 (Open Web Text)

Max Activating Token Index: 531

Text #8

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0571. Min Act: -0.1700

Data Index: 7691708 (Open Web Text)

Max Activating Token Index: 148

Text #9

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0616. Min Act: -0.1700

Data Index: 2980570 (Open Web Text)

Max Activating Token Index: 197

Text #10

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0720. Min Act: -0.1700

Data Index: 2141330 (Open Web Text)

Max Activating Token Index: 88

Text #11

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0535. Min Act: -0.1700

Data Index: 1513900 (Open Web Text)

Max Activating Token Index: 561

Text #12

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0520. Min Act: -0.1700

Data Index: 6105356 (Open Web Text)

Max Activating Token Index: 81

Text #13

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0448. Min Act: -0.1700

Data Index: 6433304 (Open Web Text)

Max Activating Token Index: 645

Text #14

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0371. Min Act: -0.1700

Data Index: 4092031 (Open Web Text)

Max Activating Token Index: 184

Text #15

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0495. Min Act: -0.1700

Data Index: 1422446 (Open Web Text)

Max Activating Token Index: 125

Text #16

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0101. Min Act: -0.1700

Data Index: 2707288 (Open Web Text)

Max Activating Token Index: 818

Text #17

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0267. Min Act: -0.1700

Data Index: 7495173 (Open Web Text)

Max Activating Token Index: 306

Text #18

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0377. Min Act: -0.1700

Data Index: 7850423 (Open Web Text)

Max Activating Token Index: 205

Text #19

Max Range: 2.1851. Min Range: -2.1851

Max Act: 2.0377. Min Act: -0.1700

Data Index: 7958280 (Open Web Text)

Max Activating Token Index: 535