Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 398 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('gpt2-small')

Text #0

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.8012. Min Act: -0.1700

Data Index: 3606723 (Open Web Text)

Max Activating Token Index: 586

Text #1

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.4153. Min Act: -0.1700

Data Index: 4294147 (Open Web Text)

Max Activating Token Index: 469

Text #2

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.3648. Min Act: -0.1700

Data Index: 6325410 (Open Web Text)

Max Activating Token Index: 37

Text #3

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.2901. Min Act: -0.1700

Data Index: 2583467 (Open Web Text)

Max Activating Token Index: 581

Text #4

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.1746. Min Act: -0.1700

Data Index: 3104367 (Open Web Text)

Max Activating Token Index: 347

Text #5

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.1823. Min Act: -0.1700

Data Index: 555358 (Open Web Text)

Max Activating Token Index: 297

Text #6

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.1792. Min Act: -0.1700

Data Index: 302207 (Open Web Text)

Max Activating Token Index: 1013

Text #7

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.0901. Min Act: -0.1700

Data Index: 2887868 (Open Web Text)

Max Activating Token Index: 393

Text #8

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.1136. Min Act: -0.1700

Data Index: 4484326 (Open Web Text)

Max Activating Token Index: 483

Text #9

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.1050. Min Act: -0.1700

Data Index: 3344124 (Open Web Text)

Max Activating Token Index: 832

Text #10

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.0093. Min Act: -0.1700

Data Index: 6476769 (Open Web Text)

Max Activating Token Index: 31

Text #11

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.0471. Min Act: -0.1700

Data Index: 7571454 (Open Web Text)

Max Activating Token Index: 693

Text #12

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9996. Min Act: -0.1700

Data Index: 4524125 (Open Web Text)

Max Activating Token Index: 875

Text #13

Max Range: 5.8012. Min Range: -5.8012

Max Act: 5.0169. Min Act: -0.1700

Data Index: 6646289 (Open Web Text)

Max Activating Token Index: 705

Text #14

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9677. Min Act: -0.1700

Data Index: 4267666 (Open Web Text)

Max Activating Token Index: 939

Text #15

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9882. Min Act: -0.1700

Data Index: 6785963 (Open Web Text)

Max Activating Token Index: 266

Text #16

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9490. Min Act: -0.1700

Data Index: 2674939 (Open Web Text)

Max Activating Token Index: 989

Text #17

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9255. Min Act: -0.1700

Data Index: 3372105 (Open Web Text)

Max Activating Token Index: 584

Text #18

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9180. Min Act: -0.1700

Data Index: 3591930 (Open Web Text)

Max Activating Token Index: 500

Text #19

Max Range: 5.8012. Min Range: -5.8012

Max Act: 4.9063. Min Act: -0.1700

Data Index: 3091052 (Open Web Text)

Max Activating Token Index: 942