Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 566 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9938. Min Act: -0.1700

Data Index: 5978867 (Open Web Text)

Max Activating Token Index: 725

Text #1

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9746. Min Act: -0.1700

Data Index: 2361199 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9746. Min Act: -0.1700

Data Index: 6612557 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9746. Min Act: -0.1700

Data Index: 232293 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9198. Min Act: -0.1700

Data Index: 5179605 (Open Web Text)

Max Activating Token Index: 1016

Text #5

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9191. Min Act: -0.1700

Data Index: 4317065 (Open Web Text)

Max Activating Token Index: 226

Text #6

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9353. Min Act: -0.1700

Data Index: 276500 (Open Web Text)

Max Activating Token Index: 628

Text #7

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9354. Min Act: -0.1700

Data Index: 7708939 (Open Web Text)

Max Activating Token Index: 721

Text #8

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9078. Min Act: -0.1700

Data Index: 6304970 (Open Web Text)

Max Activating Token Index: 246

Text #9

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9080. Min Act: -0.1700

Data Index: 1516934 (Open Web Text)

Max Activating Token Index: 946

Text #10

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8778. Min Act: -0.1700

Data Index: 1451851 (Open Web Text)

Max Activating Token Index: 172

Text #11

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9013. Min Act: -0.1700

Data Index: 4893611 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.9111. Min Act: -0.1700

Data Index: 4821390 (Open Web Text)

Max Activating Token Index: 364

Text #13

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8678. Min Act: -0.1700

Data Index: 1818464 (Open Web Text)

Max Activating Token Index: 259

Text #14

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8467. Min Act: -0.1700

Data Index: 8116049 (Open Web Text)

Max Activating Token Index: 939

Text #15

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8742. Min Act: -0.1700

Data Index: 5946478 (Open Web Text)

Max Activating Token Index: 20

Text #16

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8947. Min Act: -0.1700

Data Index: 7113939 (Open Web Text)

Max Activating Token Index: 733

Text #17

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8267. Min Act: -0.1700

Data Index: 3381195 (Open Web Text)

Max Activating Token Index: 620

Text #18

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8291. Min Act: -0.1700

Data Index: 3932404 (Open Web Text)

Max Activating Token Index: 691

Text #19

Max Range: 3.9938. Min Range: -3.9938

Max Act: 3.8027. Min Act: -0.1700

Data Index: 6632232 (Open Web Text)

Max Activating Token Index: 542