Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 179 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.7550. Min Act: -0.1700

Data Index: 5161073 (Open Web Text)

Max Activating Token Index: 466

Text #1

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.6999. Min Act: -0.1700

Data Index: 3614233 (Open Web Text)

Max Activating Token Index: 12

Text #2

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.7037. Min Act: -0.1700

Data Index: 6472822 (Open Web Text)

Max Activating Token Index: 379

Text #3

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.6421. Min Act: -0.1700

Data Index: 2799804 (Open Web Text)

Max Activating Token Index: 52

Text #4

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.6154. Min Act: -0.1700

Data Index: 7222221 (Open Web Text)

Max Activating Token Index: 270

Text #5

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5904. Min Act: -0.1700

Data Index: 414518 (Open Web Text)

Max Activating Token Index: 23

Text #6

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5974. Min Act: -0.1700

Data Index: 6798598 (Open Web Text)

Max Activating Token Index: 15

Text #7

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5635. Min Act: -0.1700

Data Index: 2991038 (Open Web Text)

Max Activating Token Index: 15

Text #8

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5743. Min Act: -0.1700

Data Index: 2063011 (Open Web Text)

Max Activating Token Index: 121

Text #9

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5304. Min Act: -0.1700

Data Index: 8462869 (Open Web Text)

Max Activating Token Index: 17

Text #10

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5657. Min Act: -0.1700

Data Index: 8023213 (Open Web Text)

Max Activating Token Index: 17

Text #11

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5542. Min Act: -0.1700

Data Index: 5364467 (Open Web Text)

Max Activating Token Index: 26

Text #12

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5565. Min Act: -0.1700

Data Index: 1335666 (Open Web Text)

Max Activating Token Index: 114

Text #13

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5436. Min Act: -0.1700

Data Index: 4323158 (Open Web Text)

Max Activating Token Index: 26

Text #14

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5638. Min Act: -0.1679

Data Index: 3896469 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.4814. Min Act: -0.1700

Data Index: 2188137 (Open Web Text)

Max Activating Token Index: 30

Text #16

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.4693. Min Act: -0.1700

Data Index: 2326755 (Open Web Text)

Max Activating Token Index: 14

Text #17

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5224. Min Act: -0.1700

Data Index: 2457075 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.4819. Min Act: -0.1700

Data Index: 2524156 (Open Web Text)

Max Activating Token Index: 28

Text #19

Max Range: 7.7550. Min Range: -7.7550

Max Act: 7.5236. Min Act: -0.1700

Data Index: 2720411 (Open Web Text)

Max Activating Token Index: 245