Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 854 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4959. Min Range: -5.4959

Max Act: 5.4959. Min Act: -0.1700

Data Index: 6518253 (Open Web Text)

Max Activating Token Index: 52

Text #1

Max Range: 5.4959. Min Range: -5.4959

Max Act: 5.4532. Min Act: -0.1700

Data Index: 7914181 (Open Web Text)

Max Activating Token Index: 25

Text #2

Max Range: 5.4959. Min Range: -5.4959

Max Act: 5.1575. Min Act: -0.1700

Data Index: 7883778 (Open Web Text)

Max Activating Token Index: 71

Text #3

Max Range: 5.4959. Min Range: -5.4959

Max Act: 5.0968. Min Act: -0.1700

Data Index: 7616154 (Open Web Text)

Max Activating Token Index: 34

Text #4

Max Range: 5.4959. Min Range: -5.4959

Max Act: 5.0492. Min Act: -0.1700

Data Index: 6466123 (Open Web Text)

Max Activating Token Index: 18

Text #5

Max Range: 5.4959. Min Range: -5.4959

Max Act: 5.0475. Min Act: -0.1700

Data Index: 2201132 (Open Web Text)

Max Activating Token Index: 21

Text #6

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9609. Min Act: -0.1700

Data Index: 2888768 (Open Web Text)

Max Activating Token Index: 20

Text #7

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9685. Min Act: -0.1700

Data Index: 1323789 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9785. Min Act: -0.1700

Data Index: 1281049 (Open Web Text)

Max Activating Token Index: 22

Text #9

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9888. Min Act: -0.1700

Data Index: 5477713 (Open Web Text)

Max Activating Token Index: 58

Text #10

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9493. Min Act: -0.1700

Data Index: 2938413 (Open Web Text)

Max Activating Token Index: 12

Text #11

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9258. Min Act: -0.1700

Data Index: 4796389 (Open Web Text)

Max Activating Token Index: 82

Text #12

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9059. Min Act: -0.1700

Data Index: 5421956 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9014. Min Act: -0.1700

Data Index: 7975567 (Open Web Text)

Max Activating Token Index: 37

Text #14

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9110. Min Act: -0.1700

Data Index: 829885 (Open Web Text)

Max Activating Token Index: 643

Text #15

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.8757. Min Act: -0.1700

Data Index: 5118248 (Open Web Text)

Max Activating Token Index: 81

Text #16

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.8600. Min Act: -0.1700

Data Index: 4456261 (Open Web Text)

Max Activating Token Index: 263

Text #17

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.9007. Min Act: -0.1700

Data Index: 1553692 (Open Web Text)

Max Activating Token Index: 530

Text #18

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.8804. Min Act: -0.1700

Data Index: 6288713 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 5.4959. Min Range: -5.4959

Max Act: 4.8753. Min Act: -0.1700

Data Index: 1850280 (Open Web Text)

Max Activating Token Index: 5