Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2087 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.8340. Min Act: -0.1700

Data Index: 192435 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.8063. Min Act: -0.1700

Data Index: 7285402 (Open Web Text)

Max Activating Token Index: 910

Text #2

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7992. Min Act: -0.1700

Data Index: 4130458 (Open Web Text)

Max Activating Token Index: 184

Text #3

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7928. Min Act: -0.1700

Data Index: 7823935 (Open Web Text)

Max Activating Token Index: 159

Text #4

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7772. Min Act: -0.1700

Data Index: 5278225 (Open Web Text)

Max Activating Token Index: 187

Text #5

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7710. Min Act: -0.1700

Data Index: 5018932 (Open Web Text)

Max Activating Token Index: 22

Text #6

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7564. Min Act: -0.1700

Data Index: 1943081 (Open Web Text)

Max Activating Token Index: 16

Text #7

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7489. Min Act: -0.1700

Data Index: 7425088 (Open Web Text)

Max Activating Token Index: 87

Text #8

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7497. Min Act: -0.1700

Data Index: 4814589 (Open Web Text)

Max Activating Token Index: 186

Text #9

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7411. Min Act: -0.1700

Data Index: 4689427 (Open Web Text)

Max Activating Token Index: 600

Text #10

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7283. Min Act: -0.1700

Data Index: 4379889 (Open Web Text)

Max Activating Token Index: 678

Text #11

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7421. Min Act: -0.1700

Data Index: 4982535 (Open Web Text)

Max Activating Token Index: 354

Text #12

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7471. Min Act: -0.1700

Data Index: 1525949 (Open Web Text)

Max Activating Token Index: 620

Text #13

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7390. Min Act: -0.1700

Data Index: 542376 (Open Web Text)

Max Activating Token Index: 386

Text #14

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7193. Min Act: -0.1700

Data Index: 6570469 (Open Web Text)

Max Activating Token Index: 615

Text #15

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7147. Min Act: -0.1700

Data Index: 1059056 (Open Web Text)

Max Activating Token Index: 747

Text #16

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7253. Min Act: -0.1700

Data Index: 6643752 (Open Web Text)

Max Activating Token Index: 380

Text #17

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7194. Min Act: -0.1700

Data Index: 3074305 (Open Web Text)

Max Activating Token Index: 611

Text #18

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7259. Min Act: -0.1700

Data Index: 2230015 (Open Web Text)

Max Activating Token Index: 514

Text #19

Max Range: 1.8340. Min Range: -1.8340

Max Act: 1.7341. Min Act: -0.1700

Data Index: 791433 (Open Web Text)

Max Activating Token Index: 522