Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2349 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.4526. Min Act: -0.1700

Data Index: 6544700 (Open Web Text)

Max Activating Token Index: 934

Text #1

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.3735. Min Act: -0.1700

Data Index: 6115062 (Open Web Text)

Max Activating Token Index: 237

Text #2

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.3474. Min Act: -0.1700

Data Index: 8643059 (Open Web Text)

Max Activating Token Index: 97

Text #3

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.3098. Min Act: -0.1700

Data Index: 2810471 (Open Web Text)

Max Activating Token Index: 718

Text #4

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.2746. Min Act: -0.1700

Data Index: 4475247 (Open Web Text)

Max Activating Token Index: 428

Text #5

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.2433. Min Act: -0.1700

Data Index: 6431097 (Open Web Text)

Max Activating Token Index: 952

Text #6

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.2051. Min Act: -0.1700

Data Index: 4509701 (Open Web Text)

Max Activating Token Index: 740

Text #7

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1861. Min Act: -0.1700

Data Index: 3308147 (Open Web Text)

Max Activating Token Index: 465

Text #8

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1783. Min Act: -0.1700

Data Index: 8107394 (Open Web Text)

Max Activating Token Index: 784

Text #9

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1510. Min Act: -0.1700

Data Index: 6740552 (Open Web Text)

Max Activating Token Index: 955

Text #10

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1749. Min Act: -0.1700

Data Index: 4061950 (Open Web Text)

Max Activating Token Index: 775

Text #11

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1504. Min Act: -0.1700

Data Index: 4868058 (Open Web Text)

Max Activating Token Index: 894

Text #12

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1520. Min Act: -0.1700

Data Index: 1554923 (Open Web Text)

Max Activating Token Index: 291

Text #13

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1687. Min Act: -0.1700

Data Index: 4154916 (Open Web Text)

Max Activating Token Index: 718

Text #14

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1166. Min Act: -0.1700

Data Index: 2264324 (Open Web Text)

Max Activating Token Index: 214

Text #15

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1175. Min Act: -0.1700

Data Index: 7886287 (Open Web Text)

Max Activating Token Index: 156

Text #16

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1134. Min Act: -0.1700

Data Index: 5854360 (Open Web Text)

Max Activating Token Index: 917

Text #17

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1066. Min Act: -0.1700

Data Index: 5685423 (Open Web Text)

Max Activating Token Index: 66

Text #18

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1296. Min Act: -0.1700

Data Index: 5400477 (Open Web Text)

Max Activating Token Index: 596

Text #19

Max Range: 3.4526. Min Range: -3.4526

Max Act: 3.1142. Min Act: -0.1700

Data Index: 1310839 (Open Web Text)

Max Activating Token Index: 717