Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 259 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.4252. Min Act: -0.1700

Data Index: 5685536 (Open Web Text)

Max Activating Token Index: 866

Text #1

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.4130. Min Act: -0.1700

Data Index: 6934991 (Open Web Text)

Max Activating Token Index: 1004

Text #2

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3975. Min Act: -0.1700

Data Index: 3440565 (Open Web Text)

Max Activating Token Index: 1019

Text #3

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.4066. Min Act: -0.1700

Data Index: 2372753 (Open Web Text)

Max Activating Token Index: 501

Text #4

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3958. Min Act: -0.1700

Data Index: 7322476 (Open Web Text)

Max Activating Token Index: 527

Text #5

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3805. Min Act: -0.1700

Data Index: 2100184 (Open Web Text)

Max Activating Token Index: 22

Text #6

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3688. Min Act: -0.1700

Data Index: 6952119 (Open Web Text)

Max Activating Token Index: 39

Text #7

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3749. Min Act: -0.1700

Data Index: 3044541 (Open Web Text)

Max Activating Token Index: 1003

Text #8

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3346. Min Act: -0.1700

Data Index: 7010159 (Open Web Text)

Max Activating Token Index: 29

Text #9

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3242. Min Act: -0.1700

Data Index: 3658957 (Open Web Text)

Max Activating Token Index: 1012

Text #10

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3523. Min Act: -0.1700

Data Index: 2462537 (Open Web Text)

Max Activating Token Index: 1020

Text #11

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3584. Min Act: -0.1700

Data Index: 8048808 (Open Web Text)

Max Activating Token Index: 28

Text #12

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3222. Min Act: -0.1700

Data Index: 7672288 (Open Web Text)

Max Activating Token Index: 960

Text #13

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3039. Min Act: -0.1700

Data Index: 4606352 (Open Web Text)

Max Activating Token Index: 28

Text #14

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3002. Min Act: -0.1700

Data Index: 2538142 (Open Web Text)

Max Activating Token Index: 710

Text #15

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.2912. Min Act: -0.1700

Data Index: 3849760 (Open Web Text)

Max Activating Token Index: 836

Text #16

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.3091. Min Act: -0.1700

Data Index: 2578717 (Open Web Text)

Max Activating Token Index: 692

Text #17

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.2769. Min Act: -0.1700

Data Index: 1244116 (Open Web Text)

Max Activating Token Index: 26

Text #18

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.2769. Min Act: -0.1700

Data Index: 4530741 (Open Web Text)

Max Activating Token Index: 43

Text #19

Max Range: 5.4252. Min Range: -5.4252

Max Act: 5.2836. Min Act: -0.1700

Data Index: 2082659 (Open Web Text)

Max Activating Token Index: 435