Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1409 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6740. Min Act: -0.1700

Data Index: 1689553 (Open Web Text)

Max Activating Token Index: 462

Text #1

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6826. Min Act: -0.1700

Data Index: 4993150 (Open Web Text)

Max Activating Token Index: 423

Text #2

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6560. Min Act: -0.1700

Data Index: 6278910 (Open Web Text)

Max Activating Token Index: 420

Text #3

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6594. Min Act: -0.1700

Data Index: 7317659 (Open Web Text)

Max Activating Token Index: 280

Text #4

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6547. Min Act: -0.1700

Data Index: 1530548 (Open Web Text)

Max Activating Token Index: 549

Text #5

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6414. Min Act: -0.1700

Data Index: 348190 (Open Web Text)

Max Activating Token Index: 252

Text #6

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6303. Min Act: -0.1700

Data Index: 6838191 (Open Web Text)

Max Activating Token Index: 23

Text #7

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6274. Min Act: -0.1700

Data Index: 8390899 (Open Web Text)

Max Activating Token Index: 15

Text #8

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6245. Min Act: -0.1700

Data Index: 7034496 (Open Web Text)

Max Activating Token Index: 211

Text #9

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6208. Min Act: -0.1700

Data Index: 8702286 (Open Web Text)

Max Activating Token Index: 548

Text #10

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6338. Min Act: -0.1700

Data Index: 1327916 (Open Web Text)

Max Activating Token Index: 230

Text #11

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6000. Min Act: -0.1700

Data Index: 2024461 (Open Web Text)

Max Activating Token Index: 225

Text #12

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6164. Min Act: -0.1700

Data Index: 4874236 (Open Web Text)

Max Activating Token Index: 746

Text #13

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6144. Min Act: -0.1700

Data Index: 1564968 (Open Web Text)

Max Activating Token Index: 726

Text #14

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6200. Min Act: -0.1700

Data Index: 5873597 (Open Web Text)

Max Activating Token Index: 585

Text #15

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6073. Min Act: -0.1700

Data Index: 5827402 (Open Web Text)

Max Activating Token Index: 84

Text #16

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6104. Min Act: -0.1700

Data Index: 7370775 (Open Web Text)

Max Activating Token Index: 625

Text #17

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.5987. Min Act: -0.1700

Data Index: 2438600 (Open Web Text)

Max Activating Token Index: 618

Text #18

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.5986. Min Act: -0.1700

Data Index: 3227362 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 1.6826. Min Range: -1.6826

Max Act: 1.6014. Min Act: -0.1700

Data Index: 4166126 (Open Web Text)

Max Activating Token Index: 173