Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 305 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.8331. Min Act: -0.1700

Data Index: 384297 (Open Web Text)

Max Activating Token Index: 475

Text #1

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.6457. Min Act: -0.1700

Data Index: 715411 (Open Web Text)

Max Activating Token Index: 31

Text #2

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.6019. Min Act: -0.1700

Data Index: 4676010 (Open Web Text)

Max Activating Token Index: 877

Text #3

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5852. Min Act: -0.1700

Data Index: 6707911 (Open Web Text)

Max Activating Token Index: 134

Text #4

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5377. Min Act: -0.1700

Data Index: 7191506 (Open Web Text)

Max Activating Token Index: 56

Text #5

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5729. Min Act: -0.1700

Data Index: 4265750 (Open Web Text)

Max Activating Token Index: 87

Text #6

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5541. Min Act: -0.1700

Data Index: 4353414 (Open Web Text)

Max Activating Token Index: 583

Text #7

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5136. Min Act: -0.1700

Data Index: 5429870 (Open Web Text)

Max Activating Token Index: 214

Text #8

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5517. Min Act: -0.1700

Data Index: 2649970 (Open Web Text)

Max Activating Token Index: 825

Text #9

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5325. Min Act: -0.1700

Data Index: 3365336 (Open Web Text)

Max Activating Token Index: 595

Text #10

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5057. Min Act: -0.1700

Data Index: 1976609 (Open Web Text)

Max Activating Token Index: 748

Text #11

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5088. Min Act: -0.1700

Data Index: 6336508 (Open Web Text)

Max Activating Token Index: 146

Text #12

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4843. Min Act: -0.1700

Data Index: 4706861 (Open Web Text)

Max Activating Token Index: 743

Text #13

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.5048. Min Act: -0.1700

Data Index: 6907319 (Open Web Text)

Max Activating Token Index: 51

Text #14

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4889. Min Act: -0.1700

Data Index: 7312105 (Open Web Text)

Max Activating Token Index: 43

Text #15

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4818. Min Act: -0.1700

Data Index: 6371511 (Open Web Text)

Max Activating Token Index: 169

Text #16

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4810. Min Act: -0.1700

Data Index: 6103227 (Open Web Text)

Max Activating Token Index: 309

Text #17

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4346. Min Act: -0.1700

Data Index: 7388168 (Open Web Text)

Max Activating Token Index: 45

Text #18

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4987. Min Act: -0.1700

Data Index: 7746836 (Open Web Text)

Max Activating Token Index: 969

Text #19

Max Range: 3.8331. Min Range: -3.8331

Max Act: 3.4559. Min Act: -0.1700

Data Index: 6753232 (Open Web Text)

Max Activating Token Index: 565