Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2198 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1421. Min Range: -4.1421

Max Act: 4.1421. Min Act: -0.1700

Data Index: 7433589 (Open Web Text)

Max Activating Token Index: 260

Text #1

Max Range: 4.1421. Min Range: -4.1421

Max Act: 4.0992. Min Act: -0.1700

Data Index: 656713 (Open Web Text)

Max Activating Token Index: 200

Text #2

Max Range: 4.1421. Min Range: -4.1421

Max Act: 4.0214. Min Act: -0.1700

Data Index: 1380398 (Open Web Text)

Max Activating Token Index: 37

Text #3

Max Range: 4.1421. Min Range: -4.1421

Max Act: 4.0608. Min Act: -0.1700

Data Index: 3507469 (Open Web Text)

Max Activating Token Index: 661

Text #4

Max Range: 4.1421. Min Range: -4.1421

Max Act: 4.0252. Min Act: -0.1700

Data Index: 324889 (Open Web Text)

Max Activating Token Index: 502

Text #5

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9392. Min Act: -0.1700

Data Index: 4989010 (Open Web Text)

Max Activating Token Index: 219

Text #6

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9774. Min Act: -0.1700

Data Index: 2338947 (Open Web Text)

Max Activating Token Index: 47

Text #7

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9155. Min Act: -0.1700

Data Index: 1459403 (Open Web Text)

Max Activating Token Index: 745

Text #8

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9441. Min Act: -0.1700

Data Index: 2823506 (Open Web Text)

Max Activating Token Index: 102

Text #9

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9102. Min Act: -0.1700

Data Index: 1774264 (Open Web Text)

Max Activating Token Index: 102

Text #10

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9231. Min Act: -0.1700

Data Index: 2507753 (Open Web Text)

Max Activating Token Index: 233

Text #11

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8904. Min Act: -0.1700

Data Index: 3684613 (Open Web Text)

Max Activating Token Index: 231

Text #12

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9095. Min Act: -0.1700

Data Index: 6892886 (Open Web Text)

Max Activating Token Index: 28

Text #13

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8808. Min Act: -0.1700

Data Index: 3058097 (Open Web Text)

Max Activating Token Index: 332

Text #14

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.9015. Min Act: -0.1700

Data Index: 8260057 (Open Web Text)

Max Activating Token Index: 244

Text #15

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8787. Min Act: -0.1700

Data Index: 2195889 (Open Web Text)

Max Activating Token Index: 69

Text #16

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8200. Min Act: -0.1700

Data Index: 6184375 (Open Web Text)

Max Activating Token Index: 22

Text #17

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8488. Min Act: -0.1700

Data Index: 5977010 (Open Web Text)

Max Activating Token Index: 565

Text #18

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8516. Min Act: -0.1700

Data Index: 1951372 (Open Web Text)

Max Activating Token Index: 587

Text #19

Max Range: 4.1421. Min Range: -4.1421

Max Act: 3.8731. Min Act: -0.1700

Data Index: 6465943 (Open Web Text)

Max Activating Token Index: 995