Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1195 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6929. Min Act: -0.1700

Data Index: 2713200 (Open Web Text)

Max Activating Token Index: 539

Text #1

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6604. Min Act: -0.1700

Data Index: 8375374 (Open Web Text)

Max Activating Token Index: 626

Text #2

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6928. Min Act: -0.1700

Data Index: 6041789 (Open Web Text)

Max Activating Token Index: 718

Text #3

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6521. Min Act: -0.1700

Data Index: 8390410 (Open Web Text)

Max Activating Token Index: 21

Text #4

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6214. Min Act: -0.1700

Data Index: 1746261 (Open Web Text)

Max Activating Token Index: 40

Text #5

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6046. Min Act: -0.1700

Data Index: 8035039 (Open Web Text)

Max Activating Token Index: 503

Text #6

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6132. Min Act: -0.1700

Data Index: 185175 (Open Web Text)

Max Activating Token Index: 883

Text #7

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6093. Min Act: -0.1700

Data Index: 8802969 (Open Web Text)

Max Activating Token Index: 189

Text #8

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5965. Min Act: -0.1700

Data Index: 8698269 (Open Web Text)

Max Activating Token Index: 924

Text #9

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5949. Min Act: -0.1700

Data Index: 5197671 (Open Web Text)

Max Activating Token Index: 501

Text #10

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5759. Min Act: -0.1700

Data Index: 2506810 (Open Web Text)

Max Activating Token Index: 22

Text #11

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5491. Min Act: -0.1700

Data Index: 2772943 (Open Web Text)

Max Activating Token Index: 607

Text #12

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.6055. Min Act: -0.1700

Data Index: 788994 (Open Web Text)

Max Activating Token Index: 862

Text #13

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5696. Min Act: -0.1700

Data Index: 2035320 (Open Web Text)

Max Activating Token Index: 583

Text #14

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5855. Min Act: -0.1700

Data Index: 4199228 (Open Web Text)

Max Activating Token Index: 351

Text #15

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5306. Min Act: -0.1700

Data Index: 3049284 (Open Web Text)

Max Activating Token Index: 768

Text #16

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5560. Min Act: -0.1700

Data Index: 3152987 (Open Web Text)

Max Activating Token Index: 145

Text #17

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5582. Min Act: -0.1700

Data Index: 3522120 (Open Web Text)

Max Activating Token Index: 162

Text #18

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5491. Min Act: -0.1700

Data Index: 6041790 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 4.6929. Min Range: -4.6929

Max Act: 4.5557. Min Act: -0.1700

Data Index: 280608 (Open Web Text)

Max Activating Token Index: 553