Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 386 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.5981. Min Act: -0.1700

Data Index: 1978019 (Open Web Text)

Max Activating Token Index: 43

Text #1

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.5295. Min Act: -0.1700

Data Index: 6048986 (Open Web Text)

Max Activating Token Index: 156

Text #2

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.5138. Min Act: -0.1700

Data Index: 7508155 (Open Web Text)

Max Activating Token Index: 893

Text #3

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4687. Min Act: -0.1700

Data Index: 8126961 (Open Web Text)

Max Activating Token Index: 20

Text #4

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4631. Min Act: -0.1700

Data Index: 5485152 (Open Web Text)

Max Activating Token Index: 413

Text #5

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4456. Min Act: -0.1700

Data Index: 7763269 (Open Web Text)

Max Activating Token Index: 92

Text #6

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4358. Min Act: -0.1700

Data Index: 6110723 (Open Web Text)

Max Activating Token Index: 212

Text #7

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4094. Min Act: -0.1700

Data Index: 2313423 (Open Web Text)

Max Activating Token Index: 355

Text #8

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4148. Min Act: -0.1700

Data Index: 2780438 (Open Web Text)

Max Activating Token Index: 33

Text #9

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4010. Min Act: -0.1700

Data Index: 4245977 (Open Web Text)

Max Activating Token Index: 59

Text #10

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4107. Min Act: -0.1700

Data Index: 265756 (Open Web Text)

Max Activating Token Index: 211

Text #11

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4232. Min Act: -0.1700

Data Index: 5327121 (Open Web Text)

Max Activating Token Index: 230

Text #12

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.4051. Min Act: -0.1700

Data Index: 4550562 (Open Web Text)

Max Activating Token Index: 100

Text #13

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3991. Min Act: -0.1700

Data Index: 6966201 (Open Web Text)

Max Activating Token Index: 33

Text #14

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3631. Min Act: -0.1700

Data Index: 6998506 (Open Web Text)

Max Activating Token Index: 78

Text #15

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3713. Min Act: -0.1700

Data Index: 785936 (Open Web Text)

Max Activating Token Index: 141

Text #16

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3690. Min Act: -0.1700

Data Index: 4532835 (Open Web Text)

Max Activating Token Index: 36

Text #17

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3903. Min Act: -0.1700

Data Index: 1416224 (Open Web Text)

Max Activating Token Index: 180

Text #18

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3960. Min Act: -0.1700

Data Index: 5091192 (Open Web Text)

Max Activating Token Index: 80

Text #19

Max Range: 5.5981. Min Range: -5.5981

Max Act: 5.3720. Min Act: -0.1700

Data Index: 5006556 (Open Web Text)

Max Activating Token Index: 21