Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2194 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.9892. Min Act: -0.1700

Data Index: 3079361 (Open Web Text)

Max Activating Token Index: 57

Text #1

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.8325. Min Act: -0.1700

Data Index: 1496074 (Open Web Text)

Max Activating Token Index: 279

Text #2

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.8280. Min Act: -0.1700

Data Index: 1157404 (Open Web Text)

Max Activating Token Index: 51

Text #3

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.8498. Min Act: -0.1700

Data Index: 4579383 (Open Web Text)

Max Activating Token Index: 321

Text #4

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.8322. Min Act: -0.1700

Data Index: 202650 (Open Web Text)

Max Activating Token Index: 450

Text #5

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7937. Min Act: -0.1700

Data Index: 7022568 (Open Web Text)

Max Activating Token Index: 216

Text #6

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7791. Min Act: -0.1700

Data Index: 8136459 (Open Web Text)

Max Activating Token Index: 110

Text #7

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7446. Min Act: -0.1700

Data Index: 1151686 (Open Web Text)

Max Activating Token Index: 769

Text #8

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7596. Min Act: -0.1700

Data Index: 3374646 (Open Web Text)

Max Activating Token Index: 615

Text #9

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7403. Min Act: -0.1700

Data Index: 4443941 (Open Web Text)

Max Activating Token Index: 711

Text #10

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7128. Min Act: -0.1700

Data Index: 4978451 (Open Web Text)

Max Activating Token Index: 608

Text #11

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7190. Min Act: -0.1700

Data Index: 3179117 (Open Web Text)

Max Activating Token Index: 291

Text #12

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7304. Min Act: -0.1700

Data Index: 2555365 (Open Web Text)

Max Activating Token Index: 911

Text #13

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7164. Min Act: -0.1700

Data Index: 7481751 (Open Web Text)

Max Activating Token Index: 137

Text #14

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7153. Min Act: -0.1700

Data Index: 5783126 (Open Web Text)

Max Activating Token Index: 497

Text #15

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7059. Min Act: -0.1700

Data Index: 133881 (Open Web Text)

Max Activating Token Index: 249

Text #16

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7165. Min Act: -0.1700

Data Index: 2086489 (Open Web Text)

Max Activating Token Index: 379

Text #17

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.7011. Min Act: -0.1700

Data Index: 7543705 (Open Web Text)

Max Activating Token Index: 781

Text #18

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.6782. Min Act: -0.1700

Data Index: 2599557 (Open Web Text)

Max Activating Token Index: 363

Text #19

Max Range: 3.9892. Min Range: -3.9892

Max Act: 3.6931. Min Act: -0.1700

Data Index: 2200371 (Open Web Text)

Max Activating Token Index: 226