Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 450 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.7497. Min Act: -0.1700

Data Index: 4653525 (Open Web Text)

Max Activating Token Index: 963

Text #1

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.7142. Min Act: -0.1700

Data Index: 1081916 (Open Web Text)

Max Activating Token Index: 696

Text #2

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.6498. Min Act: -0.1700

Data Index: 3694769 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.6312. Min Act: -0.1700

Data Index: 3659364 (Open Web Text)

Max Activating Token Index: 79

Text #4

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.5443. Min Act: -0.1700

Data Index: 3353576 (Open Web Text)

Max Activating Token Index: 8

Text #5

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.4197. Min Act: -0.1700

Data Index: 7596622 (Open Web Text)

Max Activating Token Index: 569

Text #6

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.3534. Min Act: -0.1700

Data Index: 1807608 (Open Web Text)

Max Activating Token Index: 54

Text #7

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.3289. Min Act: -0.1700

Data Index: 6532058 (Open Web Text)

Max Activating Token Index: 59

Text #8

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.3203. Min Act: -0.1700

Data Index: 2905651 (Open Web Text)

Max Activating Token Index: 117

Text #9

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.3140. Min Act: -0.1700

Data Index: 1966817 (Open Web Text)

Max Activating Token Index: 39

Text #10

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.3057. Min Act: -0.1700

Data Index: 8307709 (Open Web Text)

Max Activating Token Index: 69

Text #11

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.2365. Min Act: -0.1700

Data Index: 634059 (Open Web Text)

Max Activating Token Index: 199

Text #12

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.2121. Min Act: -0.1700

Data Index: 2987260 (Open Web Text)

Max Activating Token Index: 105

Text #13

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.2348. Min Act: -0.1700

Data Index: 8623540 (Open Web Text)

Max Activating Token Index: 800

Text #14

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.1766. Min Act: -0.1700

Data Index: 5229534 (Open Web Text)

Max Activating Token Index: 140

Text #15

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.2141. Min Act: -0.1700

Data Index: 4666051 (Open Web Text)

Max Activating Token Index: 633

Text #16

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.1415. Min Act: -0.1700

Data Index: 7070427 (Open Web Text)

Max Activating Token Index: 904

Text #17

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.1354. Min Act: -0.1700

Data Index: 181433 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.1169. Min Act: -0.1700

Data Index: 1841683 (Open Web Text)

Max Activating Token Index: 67

Text #19

Max Range: 5.7497. Min Range: -5.7497

Max Act: 5.1000. Min Act: -0.1700

Data Index: 3215302 (Open Web Text)

Max Activating Token Index: 36