Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1192 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.1483. Min Act: -0.1700

Data Index: 5753005 (Open Web Text)

Max Activating Token Index: 437

Text #1

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.1244. Min Act: -0.1700

Data Index: 380081 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0988. Min Act: -0.1700

Data Index: 6913953 (Open Web Text)

Max Activating Token Index: 151

Text #3

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0964. Min Act: -0.1700

Data Index: 4738432 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0642. Min Act: -0.1700

Data Index: 916970 (Open Web Text)

Max Activating Token Index: 78

Text #5

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0862. Min Act: -0.1700

Data Index: 5337857 (Open Web Text)

Max Activating Token Index: 21

Text #6

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0959. Min Act: -0.1700

Data Index: 3077977 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0959. Min Act: -0.1700

Data Index: 7137859 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0955. Min Act: -0.1700

Data Index: 5945461 (Open Web Text)

Max Activating Token Index: 489

Text #9

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0759. Min Act: -0.1700

Data Index: 5164382 (Open Web Text)

Max Activating Token Index: 111

Text #10

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0787. Min Act: -0.1700

Data Index: 6163430 (Open Web Text)

Max Activating Token Index: 160

Text #11

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0839. Min Act: -0.1700

Data Index: 7788864 (Open Web Text)

Max Activating Token Index: 999

Text #12

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0874. Min Act: -0.1700

Data Index: 7946110 (Open Web Text)

Max Activating Token Index: 332

Text #13

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0680. Min Act: -0.1700

Data Index: 6632557 (Open Web Text)

Max Activating Token Index: 363

Text #14

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0654. Min Act: -0.1700

Data Index: 5060983 (Open Web Text)

Max Activating Token Index: 504

Text #15

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0630. Min Act: -0.1700

Data Index: 4465404 (Open Web Text)

Max Activating Token Index: 425

Text #16

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0650. Min Act: -0.1700

Data Index: 3173458 (Open Web Text)

Max Activating Token Index: 977

Text #17

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0669. Min Act: -0.1700

Data Index: 838179 (Open Web Text)

Max Activating Token Index: 36

Text #18

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0576. Min Act: -0.1700

Data Index: 2578421 (Open Web Text)

Max Activating Token Index: 979

Text #19

Max Range: 3.1483. Min Range: -3.1483

Max Act: 3.0364. Min Act: -0.1700

Data Index: 6829444 (Open Web Text)

Max Activating Token Index: 725