Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2307 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4910. Min Act: -0.1700

Data Index: 7492544 (Open Web Text)

Max Activating Token Index: 860

Text #1

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4530. Min Act: -0.1700

Data Index: 141830 (Open Web Text)

Max Activating Token Index: 86

Text #2

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4321. Min Act: -0.1700

Data Index: 5427406 (Open Web Text)

Max Activating Token Index: 580

Text #3

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4350. Min Act: -0.1700

Data Index: 3020927 (Open Web Text)

Max Activating Token Index: 182

Text #4

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4238. Min Act: -0.1700

Data Index: 57548 (Open Web Text)

Max Activating Token Index: 635

Text #5

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4384. Min Act: -0.1700

Data Index: 8526088 (Open Web Text)

Max Activating Token Index: 1022

Text #6

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4113. Min Act: -0.1700

Data Index: 7852414 (Open Web Text)

Max Activating Token Index: 582

Text #7

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4199. Min Act: -0.1700

Data Index: 2568655 (Open Web Text)

Max Activating Token Index: 552

Text #8

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4210. Min Act: -0.1700

Data Index: 563812 (Open Web Text)

Max Activating Token Index: 199

Text #9

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3984. Min Act: -0.1700

Data Index: 8294455 (Open Web Text)

Max Activating Token Index: 769

Text #10

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.4076. Min Act: -0.1700

Data Index: 4206677 (Open Web Text)

Max Activating Token Index: 402

Text #11

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3919. Min Act: -0.1700

Data Index: 3760016 (Open Web Text)

Max Activating Token Index: 180

Text #12

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3800. Min Act: -0.1700

Data Index: 3448508 (Open Web Text)

Max Activating Token Index: 554

Text #13

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3861. Min Act: -0.1700

Data Index: 3215735 (Open Web Text)

Max Activating Token Index: 786

Text #14

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3739. Min Act: -0.1700

Data Index: 4987967 (Open Web Text)

Max Activating Token Index: 226

Text #15

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3866. Min Act: -0.1700

Data Index: 6412183 (Open Web Text)

Max Activating Token Index: 649

Text #16

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3929. Min Act: -0.1700

Data Index: 6728516 (Open Web Text)

Max Activating Token Index: 496

Text #17

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3613. Min Act: -0.1700

Data Index: 5823035 (Open Web Text)

Max Activating Token Index: 890

Text #18

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3708. Min Act: -0.1700

Data Index: 4544409 (Open Web Text)

Max Activating Token Index: 72

Text #19

Max Range: 3.4910. Min Range: -3.4910

Max Act: 3.3647. Min Act: -0.1700

Data Index: 7442987 (Open Web Text)

Max Activating Token Index: 671