Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1142 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.8032. Min Act: -0.1700

Data Index: 1931175 (Open Web Text)

Max Activating Token Index: 462

Text #1

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.5868. Min Act: -0.1700

Data Index: 7966927 (Open Web Text)

Max Activating Token Index: 248

Text #2

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.5858. Min Act: -0.1700

Data Index: 3816447 (Open Web Text)

Max Activating Token Index: 908

Text #3

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.5574. Min Act: -0.1700

Data Index: 1736849 (Open Web Text)

Max Activating Token Index: 65

Text #4

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.4903. Min Act: -0.1700

Data Index: 4383678 (Open Web Text)

Max Activating Token Index: 844

Text #5

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.5262. Min Act: -0.1700

Data Index: 3954158 (Open Web Text)

Max Activating Token Index: 861

Text #6

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.4899. Min Act: -0.1700

Data Index: 8323964 (Open Web Text)

Max Activating Token Index: 51

Text #7

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.4699. Min Act: -0.1700

Data Index: 2900373 (Open Web Text)

Max Activating Token Index: 275

Text #8

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.4530. Min Act: -0.1700

Data Index: 8495318 (Open Web Text)

Max Activating Token Index: 504

Text #9

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.4343. Min Act: -0.1700

Data Index: 1254608 (Open Web Text)

Max Activating Token Index: 44

Text #10

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.4088. Min Act: -0.1700

Data Index: 1771722 (Open Web Text)

Max Activating Token Index: 501

Text #11

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3956. Min Act: -0.1700

Data Index: 8027508 (Open Web Text)

Max Activating Token Index: 267

Text #12

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3627. Min Act: -0.1700

Data Index: 5972504 (Open Web Text)

Max Activating Token Index: 117

Text #13

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3524. Min Act: -0.1700

Data Index: 424893 (Open Web Text)

Max Activating Token Index: 352

Text #14

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3719. Min Act: -0.1700

Data Index: 7499588 (Open Web Text)

Max Activating Token Index: 304

Text #15

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3402. Min Act: -0.1700

Data Index: 8484095 (Open Web Text)

Max Activating Token Index: 115

Text #16

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3485. Min Act: -0.1700

Data Index: 7287951 (Open Web Text)

Max Activating Token Index: 197

Text #17

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3069. Min Act: -0.1700

Data Index: 2168857 (Open Web Text)

Max Activating Token Index: 209

Text #18

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.2975. Min Act: -0.1700

Data Index: 2384751 (Open Web Text)

Max Activating Token Index: 190

Text #19

Max Range: 4.8032. Min Range: -4.8032

Max Act: 4.3109. Min Act: -0.1700

Data Index: 7910877 (Open Web Text)

Max Activating Token Index: 861