Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1029 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.5443. Min Act: -0.1700

Data Index: 7399688 (Open Web Text)

Max Activating Token Index: 37

Text #1

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.5275. Min Act: -0.1700

Data Index: 6885778 (Open Web Text)

Max Activating Token Index: 36

Text #2

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4857. Min Act: -0.1700

Data Index: 7855414 (Open Web Text)

Max Activating Token Index: 221

Text #3

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4525. Min Act: -0.1700

Data Index: 7782656 (Open Web Text)

Max Activating Token Index: 267

Text #4

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4564. Min Act: -0.1700

Data Index: 5267592 (Open Web Text)

Max Activating Token Index: 252

Text #5

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4632. Min Act: -0.1700

Data Index: 1099697 (Open Web Text)

Max Activating Token Index: 60

Text #6

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4721. Min Act: -0.1700

Data Index: 2797784 (Open Web Text)

Max Activating Token Index: 59

Text #7

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4444. Min Act: -0.1700

Data Index: 849638 (Open Web Text)

Max Activating Token Index: 113

Text #8

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4174. Min Act: -0.1700

Data Index: 2862700 (Open Web Text)

Max Activating Token Index: 151

Text #9

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4258. Min Act: -0.1700

Data Index: 2259199 (Open Web Text)

Max Activating Token Index: 486

Text #10

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4078. Min Act: -0.1700

Data Index: 4200770 (Open Web Text)

Max Activating Token Index: 55

Text #11

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3879. Min Act: -0.1700

Data Index: 8156310 (Open Web Text)

Max Activating Token Index: 326

Text #12

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4029. Min Act: -0.1700

Data Index: 4200403 (Open Web Text)

Max Activating Token Index: 641

Text #13

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.4061. Min Act: -0.1700

Data Index: 5175506 (Open Web Text)

Max Activating Token Index: 122

Text #14

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3739. Min Act: -0.1700

Data Index: 4814402 (Open Web Text)

Max Activating Token Index: 37

Text #15

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3524. Min Act: -0.1700

Data Index: 2101554 (Open Web Text)

Max Activating Token Index: 62

Text #16

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3772. Min Act: -0.1700

Data Index: 1450378 (Open Web Text)

Max Activating Token Index: 122

Text #17

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3651. Min Act: -0.1700

Data Index: 4979500 (Open Web Text)

Max Activating Token Index: 11

Text #18

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3790. Min Act: -0.1700

Data Index: 1826520 (Open Web Text)

Max Activating Token Index: 259

Text #19

Max Range: 4.5443. Min Range: -4.5443

Max Act: 4.3842. Min Act: -0.1700

Data Index: 3158027 (Open Web Text)

Max Activating Token Index: 649