Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1401 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0059. Min Range: -4.0059

Max Act: 4.0059. Min Act: -0.1700

Data Index: 8471975 (Open Web Text)

Max Activating Token Index: 94

Text #1

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9554. Min Act: -0.1700

Data Index: 1982301 (Open Web Text)

Max Activating Token Index: 535

Text #2

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9271. Min Act: -0.1700

Data Index: 8224179 (Open Web Text)

Max Activating Token Index: 227

Text #3

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9370. Min Act: -0.1700

Data Index: 1103353 (Open Web Text)

Max Activating Token Index: 385

Text #4

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9443. Min Act: -0.1700

Data Index: 1377329 (Open Web Text)

Max Activating Token Index: 619

Text #5

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9235. Min Act: -0.1700

Data Index: 6119760 (Open Web Text)

Max Activating Token Index: 436

Text #6

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8698. Min Act: -0.1700

Data Index: 7334604 (Open Web Text)

Max Activating Token Index: 428

Text #7

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8969. Min Act: -0.1700

Data Index: 6529770 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9002. Min Act: -0.1700

Data Index: 5452979 (Open Web Text)

Max Activating Token Index: 497

Text #9

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8920. Min Act: -0.1700

Data Index: 1377324 (Open Web Text)

Max Activating Token Index: 495

Text #10

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8904. Min Act: -0.1700

Data Index: 3110594 (Open Web Text)

Max Activating Token Index: 274

Text #11

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8702. Min Act: -0.1700

Data Index: 2040202 (Open Web Text)

Max Activating Token Index: 132

Text #12

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8804. Min Act: -0.1700

Data Index: 1582004 (Open Web Text)

Max Activating Token Index: 157

Text #13

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8844. Min Act: -0.1700

Data Index: 2164932 (Open Web Text)

Max Activating Token Index: 705

Text #14

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8989. Min Act: -0.1700

Data Index: 1077253 (Open Web Text)

Max Activating Token Index: 696

Text #15

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8704. Min Act: -0.1700

Data Index: 6465116 (Open Web Text)

Max Activating Token Index: 853

Text #16

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8870. Min Act: -0.1700

Data Index: 6577921 (Open Web Text)

Max Activating Token Index: 666

Text #17

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8778. Min Act: -0.1700

Data Index: 6880047 (Open Web Text)

Max Activating Token Index: 1003

Text #18

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.9014. Min Act: -0.1700

Data Index: 6946250 (Open Web Text)

Max Activating Token Index: 100

Text #19

Max Range: 4.0059. Min Range: -4.0059

Max Act: 3.8617. Min Act: -0.1700

Data Index: 2726249 (Open Web Text)

Max Activating Token Index: 758