Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3041 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2103. Min Range: -4.2103

Max Act: 4.2103. Min Act: -0.1700

Data Index: 6603103 (Open Web Text)

Max Activating Token Index: 997

Text #1

Max Range: 4.2103. Min Range: -4.2103

Max Act: 4.1938. Min Act: -0.1700

Data Index: 484849 (Open Web Text)

Max Activating Token Index: 893

Text #2

Max Range: 4.2103. Min Range: -4.2103

Max Act: 4.1584. Min Act: -0.1700

Data Index: 6597742 (Open Web Text)

Max Activating Token Index: 999

Text #3

Max Range: 4.2103. Min Range: -4.2103

Max Act: 4.0636. Min Act: -0.1700

Data Index: 7213375 (Open Web Text)

Max Activating Token Index: 339

Text #4

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9494. Min Act: -0.1700

Data Index: 2789796 (Open Web Text)

Max Activating Token Index: 348

Text #5

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9566. Min Act: -0.1700

Data Index: 7837069 (Open Web Text)

Max Activating Token Index: 717

Text #6

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9343. Min Act: -0.1700

Data Index: 4431710 (Open Web Text)

Max Activating Token Index: 956

Text #7

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9655. Min Act: -0.1700

Data Index: 7240243 (Open Web Text)

Max Activating Token Index: 674

Text #8

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.8191. Min Act: -0.1700

Data Index: 1725109 (Open Web Text)

Max Activating Token Index: 973

Text #9

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9463. Min Act: -0.1700

Data Index: 8560857 (Open Web Text)

Max Activating Token Index: 848

Text #10

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9843. Min Act: -0.1700

Data Index: 8442933 (Open Web Text)

Max Activating Token Index: 582

Text #11

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.8365. Min Act: -0.1700

Data Index: 1868865 (Open Web Text)

Max Activating Token Index: 944

Text #12

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9179. Min Act: -0.1700

Data Index: 6211813 (Open Web Text)

Max Activating Token Index: 1018

Text #13

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9051. Min Act: -0.1700

Data Index: 7521590 (Open Web Text)

Max Activating Token Index: 744

Text #14

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.8806. Min Act: -0.1700

Data Index: 4507649 (Open Web Text)

Max Activating Token Index: 401

Text #15

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.8296. Min Act: -0.1700

Data Index: 4912031 (Open Web Text)

Max Activating Token Index: 516

Text #16

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9267. Min Act: -0.1700

Data Index: 5787582 (Open Web Text)

Max Activating Token Index: 1021

Text #17

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.8848. Min Act: -0.1700

Data Index: 2272595 (Open Web Text)

Max Activating Token Index: 1007

Text #18

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.9222. Min Act: -0.1700

Data Index: 6742584 (Open Web Text)

Max Activating Token Index: 848

Text #19

Max Range: 4.2103. Min Range: -4.2103

Max Act: 3.8525. Min Act: -0.1700

Data Index: 5161823 (Open Web Text)

Max Activating Token Index: 955