Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 184 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1237. Min Range: -4.1237

Max Act: 4.1237. Min Act: -0.1700

Data Index: 6613903 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 4.1237. Min Range: -4.1237

Max Act: 4.0496. Min Act: -0.1700

Data Index: 5344823 (Open Web Text)

Max Activating Token Index: 433

Text #2

Max Range: 4.1237. Min Range: -4.1237

Max Act: 4.0465. Min Act: -0.1700

Data Index: 6631113 (Open Web Text)

Max Activating Token Index: 13

Text #3

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9573. Min Act: -0.1700

Data Index: 1168230 (Open Web Text)

Max Activating Token Index: 394

Text #4

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9586. Min Act: -0.1700

Data Index: 7711081 (Open Web Text)

Max Activating Token Index: 14

Text #5

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9486. Min Act: -0.1700

Data Index: 4700089 (Open Web Text)

Max Activating Token Index: 768

Text #6

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9368. Min Act: -0.1700

Data Index: 1716872 (Open Web Text)

Max Activating Token Index: 204

Text #7

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9605. Min Act: -0.1700

Data Index: 4700102 (Open Web Text)

Max Activating Token Index: 481

Text #8

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9303. Min Act: -0.1700

Data Index: 2709766 (Open Web Text)

Max Activating Token Index: 462

Text #9

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9202. Min Act: -0.1700

Data Index: 5359353 (Open Web Text)

Max Activating Token Index: 274

Text #10

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9352. Min Act: -0.1700

Data Index: 5425818 (Open Web Text)

Max Activating Token Index: 783

Text #11

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9432. Min Act: -0.1700

Data Index: 6191623 (Open Web Text)

Max Activating Token Index: 13

Text #12

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9215. Min Act: -0.1700

Data Index: 5185240 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9123. Min Act: -0.1700

Data Index: 1444461 (Open Web Text)

Max Activating Token Index: 407

Text #14

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9112. Min Act: -0.1700

Data Index: 2596389 (Open Web Text)

Max Activating Token Index: 331

Text #15

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9122. Min Act: -0.1700

Data Index: 6833603 (Open Web Text)

Max Activating Token Index: 145

Text #16

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9071. Min Act: -0.1700

Data Index: 5490853 (Open Web Text)

Max Activating Token Index: 33

Text #17

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9160. Min Act: -0.1700

Data Index: 5020184 (Open Web Text)

Max Activating Token Index: 210

Text #18

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9171. Min Act: -0.1700

Data Index: 1028969 (Open Web Text)

Max Activating Token Index: 13

Text #19

Max Range: 4.1237. Min Range: -4.1237

Max Act: 3.9060. Min Act: -0.1700

Data Index: 3513016 (Open Web Text)

Max Activating Token Index: 15