Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1955 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.7787. Min Act: -0.1700

Data Index: 5499635 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.7840. Min Act: -0.1700

Data Index: 3386084 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.7207. Min Act: -0.1700

Data Index: 2006417 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.7004. Min Act: -0.1700

Data Index: 2055886 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6853. Min Act: -0.1700

Data Index: 7608932 (Open Web Text)

Max Activating Token Index: 13

Text #5

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6946. Min Act: -0.1700

Data Index: 1102600 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6484. Min Act: -0.1700

Data Index: 6184821 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6601. Min Act: -0.1700

Data Index: 6791111 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6557. Min Act: -0.1700

Data Index: 8581777 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6351. Min Act: -0.1700

Data Index: 4251965 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6004. Min Act: -0.1700

Data Index: 4578802 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5695. Min Act: -0.1700

Data Index: 5132357 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5445. Min Act: -0.1700

Data Index: 5105508 (Open Web Text)

Max Activating Token Index: 15

Text #13

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5849. Min Act: -0.1700

Data Index: 2809781 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5533. Min Act: -0.1700

Data Index: 4673175 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5905. Min Act: -0.1700

Data Index: 2610608 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5707. Min Act: -0.1700

Data Index: 2607813 (Open Web Text)

Max Activating Token Index: 12

Text #17

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.6014. Min Act: -0.1700

Data Index: 4149363 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5627. Min Act: -0.1700

Data Index: 1266179 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 6.7840. Min Range: -6.7840

Max Act: 6.5881. Min Act: -0.1700

Data Index: 3356563 (Open Web Text)

Max Activating Token Index: 6