Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2468 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.9410. Min Act: -0.1700

Data Index: 1057295 (Open Web Text)

Max Activating Token Index: 555

Text #1

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.9242. Min Act: -0.1700

Data Index: 3299881 (Open Web Text)

Max Activating Token Index: 809

Text #2

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8622. Min Act: -0.1700

Data Index: 1004097 (Open Web Text)

Max Activating Token Index: 715

Text #3

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8183. Min Act: -0.1700

Data Index: 1612274 (Open Web Text)

Max Activating Token Index: 367

Text #4

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8526. Min Act: -0.1700

Data Index: 2867272 (Open Web Text)

Max Activating Token Index: 84

Text #5

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7515. Min Act: -0.1700

Data Index: 8694163 (Open Web Text)

Max Activating Token Index: 527

Text #6

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8052. Min Act: -0.1700

Data Index: 6272928 (Open Web Text)

Max Activating Token Index: 457

Text #7

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7598. Min Act: -0.1700

Data Index: 6201392 (Open Web Text)

Max Activating Token Index: 928

Text #8

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8212. Min Act: -0.1700

Data Index: 1085636 (Open Web Text)

Max Activating Token Index: 114

Text #9

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8048. Min Act: -0.1700

Data Index: 7626478 (Open Web Text)

Max Activating Token Index: 421

Text #10

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7577. Min Act: -0.1700

Data Index: 8560491 (Open Web Text)

Max Activating Token Index: 229

Text #11

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7339. Min Act: -0.1700

Data Index: 3524017 (Open Web Text)

Max Activating Token Index: 148

Text #12

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8327. Min Act: -0.1700

Data Index: 8648931 (Open Web Text)

Max Activating Token Index: 544

Text #13

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.8042. Min Act: -0.1700

Data Index: 5815720 (Open Web Text)

Max Activating Token Index: 169

Text #14

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7413. Min Act: -0.1700

Data Index: 5663103 (Open Web Text)

Max Activating Token Index: 472

Text #15

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7449. Min Act: -0.1700

Data Index: 1532079 (Open Web Text)

Max Activating Token Index: 52

Text #16

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7171. Min Act: -0.1700

Data Index: 1638400 (Open Web Text)

Max Activating Token Index: 370

Text #17

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7042. Min Act: -0.1700

Data Index: 6498591 (Open Web Text)

Max Activating Token Index: 185

Text #18

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.6830. Min Act: -0.1700

Data Index: 2989740 (Open Web Text)

Max Activating Token Index: 301

Text #19

Max Range: 4.9410. Min Range: -4.9410

Max Act: 4.7114. Min Act: -0.1700

Data Index: 1962018 (Open Web Text)

Max Activating Token Index: 927