Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 995 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0043. Min Range: -5.0043

Max Act: 5.0043. Min Act: -0.1700

Data Index: 5755481 (Open Web Text)

Max Activating Token Index: 88

Text #1

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.9447. Min Act: -0.1700

Data Index: 5755479 (Open Web Text)

Max Activating Token Index: 42

Text #2

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.9540. Min Act: -0.1700

Data Index: 5774737 (Open Web Text)

Max Activating Token Index: 191

Text #3

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.9420. Min Act: -0.1700

Data Index: 5755478 (Open Web Text)

Max Activating Token Index: 413

Text #4

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.9166. Min Act: -0.1700

Data Index: 5774735 (Open Web Text)

Max Activating Token Index: 73

Text #5

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.9248. Min Act: -0.1700

Data Index: 5774734 (Open Web Text)

Max Activating Token Index: 76

Text #6

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8804. Min Act: -0.1700

Data Index: 5774727 (Open Web Text)

Max Activating Token Index: 37

Text #7

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.9001. Min Act: -0.1700

Data Index: 8781154 (Open Web Text)

Max Activating Token Index: 562

Text #8

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8416. Min Act: -0.1700

Data Index: 5755484 (Open Web Text)

Max Activating Token Index: 354

Text #9

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8174. Min Act: -0.1700

Data Index: 5774731 (Open Web Text)

Max Activating Token Index: 42

Text #10

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8409. Min Act: -0.1700

Data Index: 1443517 (Open Web Text)

Max Activating Token Index: 131

Text #11

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.7992. Min Act: -0.1700

Data Index: 5774730 (Open Web Text)

Max Activating Token Index: 393

Text #12

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8132. Min Act: -0.1700

Data Index: 5755477 (Open Web Text)

Max Activating Token Index: 356

Text #13

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8156. Min Act: -0.1700

Data Index: 5774732 (Open Web Text)

Max Activating Token Index: 184

Text #14

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.7528. Min Act: -0.1700

Data Index: 5755456 (Open Web Text)

Max Activating Token Index: 112

Text #15

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.7786. Min Act: -0.1700

Data Index: 1998145 (Open Web Text)

Max Activating Token Index: 391

Text #16

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.8136. Min Act: -0.1700

Data Index: 4353619 (Open Web Text)

Max Activating Token Index: 637

Text #17

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.7850. Min Act: -0.1700

Data Index: 5774718 (Open Web Text)

Max Activating Token Index: 26

Text #18

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.7807. Min Act: -0.1700

Data Index: 3010689 (Open Web Text)

Max Activating Token Index: 120

Text #19

Max Range: 5.0043. Min Range: -5.0043

Max Act: 4.7653. Min Act: -0.1700

Data Index: 5774720 (Open Web Text)

Max Activating Token Index: 49