Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 291 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0549. Min Act: -0.1700

Data Index: 4756492 (Open Web Text)

Max Activating Token Index: 381

Text #1

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0355. Min Act: -0.1700

Data Index: 2692208 (Open Web Text)

Max Activating Token Index: 855

Text #2

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0437. Min Act: -0.1700

Data Index: 6427268 (Open Web Text)

Max Activating Token Index: 213

Text #3

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0465. Min Act: -0.1700

Data Index: 5054944 (Open Web Text)

Max Activating Token Index: 776

Text #4

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0125. Min Act: -0.1700

Data Index: 2321079 (Open Web Text)

Max Activating Token Index: 614

Text #5

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0096. Min Act: -0.1700

Data Index: 906720 (Open Web Text)

Max Activating Token Index: 114

Text #6

Max Range: 3.0549. Min Range: -3.0549

Max Act: 3.0323. Min Act: -0.1700

Data Index: 2971769 (Open Web Text)

Max Activating Token Index: 202

Text #7

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9993. Min Act: -0.1700

Data Index: 5783826 (Open Web Text)

Max Activating Token Index: 112

Text #8

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9759. Min Act: -0.1700

Data Index: 5751710 (Open Web Text)

Max Activating Token Index: 300

Text #9

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9686. Min Act: -0.1700

Data Index: 2054575 (Open Web Text)

Max Activating Token Index: 111

Text #10

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9719. Min Act: -0.1700

Data Index: 4789295 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9706. Min Act: -0.1700

Data Index: 8646177 (Open Web Text)

Max Activating Token Index: 62

Text #12

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9475. Min Act: -0.1700

Data Index: 6013177 (Open Web Text)

Max Activating Token Index: 843

Text #13

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9666. Min Act: -0.1700

Data Index: 7915317 (Open Web Text)

Max Activating Token Index: 474

Text #14

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9299. Min Act: -0.1700

Data Index: 6638972 (Open Web Text)

Max Activating Token Index: 21

Text #15

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9465. Min Act: -0.1700

Data Index: 1774021 (Open Web Text)

Max Activating Token Index: 352

Text #16

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9308. Min Act: -0.1700

Data Index: 4962193 (Open Web Text)

Max Activating Token Index: 126

Text #17

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9372. Min Act: -0.1700

Data Index: 279632 (Open Web Text)

Max Activating Token Index: 56

Text #18

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9377. Min Act: -0.1700

Data Index: 7356218 (Open Web Text)

Max Activating Token Index: 70

Text #19

Max Range: 3.0549. Min Range: -3.0549

Max Act: 2.9484. Min Act: -0.1700

Data Index: 5392663 (Open Web Text)

Max Activating Token Index: 76