Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2809 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.6130. Min Act: -0.1700

Data Index: 3943673 (Open Web Text)

Max Activating Token Index: 730

Text #1

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.5451. Min Act: -0.1700

Data Index: 6061918 (Open Web Text)

Max Activating Token Index: 790

Text #2

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.5142. Min Act: -0.1700

Data Index: 4411052 (Open Web Text)

Max Activating Token Index: 271

Text #3

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4998. Min Act: -0.1700

Data Index: 5912702 (Open Web Text)

Max Activating Token Index: 462

Text #4

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4811. Min Act: -0.1700

Data Index: 7644109 (Open Web Text)

Max Activating Token Index: 806

Text #5

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4771. Min Act: -0.1700

Data Index: 387220 (Open Web Text)

Max Activating Token Index: 942

Text #6

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4340. Min Act: -0.1700

Data Index: 2006731 (Open Web Text)

Max Activating Token Index: 283

Text #7

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4123. Min Act: -0.1700

Data Index: 2925238 (Open Web Text)

Max Activating Token Index: 574

Text #8

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4331. Min Act: -0.1700

Data Index: 3680950 (Open Web Text)

Max Activating Token Index: 543

Text #9

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4155. Min Act: -0.1700

Data Index: 443723 (Open Web Text)

Max Activating Token Index: 62

Text #10

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.4371. Min Act: -0.1700

Data Index: 3862166 (Open Web Text)

Max Activating Token Index: 845

Text #11

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3914. Min Act: -0.1700

Data Index: 7180817 (Open Web Text)

Max Activating Token Index: 102

Text #12

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3840. Min Act: -0.1700

Data Index: 1013554 (Open Web Text)

Max Activating Token Index: 356

Text #13

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3798. Min Act: -0.1700

Data Index: 4443810 (Open Web Text)

Max Activating Token Index: 460

Text #14

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3659. Min Act: -0.1700

Data Index: 7500034 (Open Web Text)

Max Activating Token Index: 111

Text #15

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3505. Min Act: -0.1700

Data Index: 8442968 (Open Web Text)

Max Activating Token Index: 139

Text #16

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3592. Min Act: -0.1700

Data Index: 7650670 (Open Web Text)

Max Activating Token Index: 769

Text #17

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3205. Min Act: -0.1700

Data Index: 6838957 (Open Web Text)

Max Activating Token Index: 920

Text #18

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3497. Min Act: -0.1700

Data Index: 518825 (Open Web Text)

Max Activating Token Index: 460

Text #19

Max Range: 4.6130. Min Range: -4.6130

Max Act: 4.3594. Min Act: -0.1700

Data Index: 1955331 (Open Web Text)

Max Activating Token Index: 327