Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1601 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.5341. Min Act: -0.1700

Data Index: 5837626 (Open Web Text)

Max Activating Token Index: 896

Text #1

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.3882. Min Act: -0.1700

Data Index: 2125894 (Open Web Text)

Max Activating Token Index: 317

Text #2

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2999. Min Act: -0.1700

Data Index: 209582 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2977. Min Act: -0.1700

Data Index: 401726 (Open Web Text)

Max Activating Token Index: 202

Text #4

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2663. Min Act: -0.1700

Data Index: 8312721 (Open Web Text)

Max Activating Token Index: 699

Text #5

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2860. Min Act: -0.1700

Data Index: 8787632 (Open Web Text)

Max Activating Token Index: 96

Text #6

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2661. Min Act: -0.1700

Data Index: 1184275 (Open Web Text)

Max Activating Token Index: 417

Text #7

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2328. Min Act: -0.1700

Data Index: 7187426 (Open Web Text)

Max Activating Token Index: 616

Text #8

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2148. Min Act: -0.1700

Data Index: 7264526 (Open Web Text)

Max Activating Token Index: 743

Text #9

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2171. Min Act: -0.1700

Data Index: 8693084 (Open Web Text)

Max Activating Token Index: 357

Text #10

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2253. Min Act: -0.1700

Data Index: 6093700 (Open Web Text)

Max Activating Token Index: 638

Text #11

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.1863. Min Act: -0.1700

Data Index: 3401583 (Open Web Text)

Max Activating Token Index: 575

Text #12

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2137. Min Act: -0.1700

Data Index: 5103477 (Open Web Text)

Max Activating Token Index: 283

Text #13

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2086. Min Act: -0.1700

Data Index: 807825 (Open Web Text)

Max Activating Token Index: 405

Text #14

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2371. Min Act: -0.1700

Data Index: 1723086 (Open Web Text)

Max Activating Token Index: 367

Text #15

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2109. Min Act: -0.1700

Data Index: 8693086 (Open Web Text)

Max Activating Token Index: 215

Text #16

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2169. Min Act: -0.1700

Data Index: 2999410 (Open Web Text)

Max Activating Token Index: 254

Text #17

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2126. Min Act: -0.1700

Data Index: 2821538 (Open Web Text)

Max Activating Token Index: 154

Text #18

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.2020. Min Act: -0.1700

Data Index: 429644 (Open Web Text)

Max Activating Token Index: 115

Text #19

Max Range: 3.5341. Min Range: -3.5341

Max Act: 3.1966. Min Act: -0.1700

Data Index: 4278727 (Open Web Text)

Max Activating Token Index: 897