Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 906 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1266. Min Range: -4.1266

Max Act: 4.1266. Min Act: -0.1700

Data Index: 6162335 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.1266. Min Range: -4.1266

Max Act: 4.0161. Min Act: -0.1700

Data Index: 68582 (Open Web Text)

Max Activating Token Index: 172

Text #2

Max Range: 4.1266. Min Range: -4.1266

Max Act: 4.0159. Min Act: -0.1700

Data Index: 824641 (Open Web Text)

Max Activating Token Index: 482

Text #3

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9986. Min Act: -0.1700

Data Index: 8552660 (Open Web Text)

Max Activating Token Index: 11

Text #4

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9807. Min Act: -0.1700

Data Index: 2288119 (Open Web Text)

Max Activating Token Index: 614

Text #5

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9842. Min Act: -0.1700

Data Index: 5313695 (Open Web Text)

Max Activating Token Index: 21

Text #6

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9723. Min Act: -0.1700

Data Index: 585492 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9837. Min Act: -0.1700

Data Index: 3348372 (Open Web Text)

Max Activating Token Index: 27

Text #8

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9784. Min Act: -0.1700

Data Index: 2005784 (Open Web Text)

Max Activating Token Index: 11

Text #9

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9653. Min Act: -0.1700

Data Index: 7579325 (Open Web Text)

Max Activating Token Index: 842

Text #10

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9477. Min Act: -0.1700

Data Index: 7247804 (Open Web Text)

Max Activating Token Index: 445

Text #11

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9433. Min Act: -0.1700

Data Index: 4453167 (Open Web Text)

Max Activating Token Index: 19

Text #12

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9607. Min Act: -0.1700

Data Index: 5526507 (Open Web Text)

Max Activating Token Index: 678

Text #13

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9568. Min Act: -0.1700

Data Index: 4709393 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9532. Min Act: -0.1700

Data Index: 1635304 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9571. Min Act: -0.1700

Data Index: 389187 (Open Web Text)

Max Activating Token Index: 401

Text #16

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9271. Min Act: -0.1700

Data Index: 3807189 (Open Web Text)

Max Activating Token Index: 503

Text #17

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9246. Min Act: -0.1700

Data Index: 4321284 (Open Web Text)

Max Activating Token Index: 647

Text #18

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9318. Min Act: -0.1700

Data Index: 2765354 (Open Web Text)

Max Activating Token Index: 68

Text #19

Max Range: 4.1266. Min Range: -4.1266

Max Act: 3.9082. Min Act: -0.1700

Data Index: 1468262 (Open Web Text)

Max Activating Token Index: 12