Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1995 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.8656. Min Act: -0.1700

Data Index: 7587267 (Open Web Text)

Max Activating Token Index: 17

Text #1

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.7705. Min Act: -0.1700

Data Index: 3407920 (Open Web Text)

Max Activating Token Index: 879

Text #2

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.7331. Min Act: -0.1700

Data Index: 1771384 (Open Web Text)

Max Activating Token Index: 181

Text #3

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.7423. Min Act: -0.1700

Data Index: 7800495 (Open Web Text)

Max Activating Token Index: 182

Text #4

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.7233. Min Act: -0.1700

Data Index: 8634213 (Open Web Text)

Max Activating Token Index: 656

Text #5

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.7376. Min Act: -0.1700

Data Index: 4582406 (Open Web Text)

Max Activating Token Index: 973

Text #6

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6807. Min Act: -0.1700

Data Index: 5870389 (Open Web Text)

Max Activating Token Index: 474

Text #7

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6725. Min Act: -0.1700

Data Index: 5448477 (Open Web Text)

Max Activating Token Index: 872

Text #8

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6731. Min Act: -0.1700

Data Index: 1767022 (Open Web Text)

Max Activating Token Index: 706

Text #9

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6896. Min Act: -0.1700

Data Index: 4351983 (Open Web Text)

Max Activating Token Index: 1017

Text #10

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6673. Min Act: -0.1700

Data Index: 7005701 (Open Web Text)

Max Activating Token Index: 159

Text #11

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6593. Min Act: -0.1700

Data Index: 681348 (Open Web Text)

Max Activating Token Index: 295

Text #12

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6440. Min Act: -0.1700

Data Index: 3462663 (Open Web Text)

Max Activating Token Index: 334

Text #13

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6228. Min Act: -0.1700

Data Index: 2435632 (Open Web Text)

Max Activating Token Index: 19

Text #14

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6435. Min Act: -0.1700

Data Index: 45247 (Open Web Text)

Max Activating Token Index: 31

Text #15

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6436. Min Act: -0.1700

Data Index: 2285549 (Open Web Text)

Max Activating Token Index: 946

Text #16

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6456. Min Act: -0.1700

Data Index: 3348494 (Open Web Text)

Max Activating Token Index: 883

Text #17

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6148. Min Act: -0.1700

Data Index: 5116932 (Open Web Text)

Max Activating Token Index: 19

Text #18

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6544. Min Act: -0.1700

Data Index: 5619850 (Open Web Text)

Max Activating Token Index: 962

Text #19

Max Range: 3.8656. Min Range: -3.8656

Max Act: 3.6226. Min Act: -0.1700

Data Index: 1038948 (Open Web Text)

Max Activating Token Index: 391