Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 784 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.5172. Min Act: -0.1700

Data Index: 3617379 (Open Web Text)

Max Activating Token Index: 342

Text #1

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.3093. Min Act: -0.1700

Data Index: 4469127 (Open Web Text)

Max Activating Token Index: 662

Text #2

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.2961. Min Act: -0.1700

Data Index: 3134459 (Open Web Text)

Max Activating Token Index: 370

Text #3

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.2765. Min Act: -0.1700

Data Index: 6885785 (Open Web Text)

Max Activating Token Index: 863

Text #4

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.2477. Min Act: -0.1700

Data Index: 8681968 (Open Web Text)

Max Activating Token Index: 226

Text #5

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.2131. Min Act: -0.1700

Data Index: 7484202 (Open Web Text)

Max Activating Token Index: 106

Text #6

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1845. Min Act: -0.1700

Data Index: 2047270 (Open Web Text)

Max Activating Token Index: 777

Text #7

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1617. Min Act: -0.1700

Data Index: 8761245 (Open Web Text)

Max Activating Token Index: 625

Text #8

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1586. Min Act: -0.1700

Data Index: 3119781 (Open Web Text)

Max Activating Token Index: 181

Text #9

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1488. Min Act: -0.1700

Data Index: 8454969 (Open Web Text)

Max Activating Token Index: 629

Text #10

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1535. Min Act: -0.1700

Data Index: 4035512 (Open Web Text)

Max Activating Token Index: 660

Text #11

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1460. Min Act: -0.1700

Data Index: 1245192 (Open Web Text)

Max Activating Token Index: 343

Text #12

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1649. Min Act: -0.1700

Data Index: 6325492 (Open Web Text)

Max Activating Token Index: 522

Text #13

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1767. Min Act: -0.1700

Data Index: 5961621 (Open Web Text)

Max Activating Token Index: 709

Text #14

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1707. Min Act: -0.1700

Data Index: 3009493 (Open Web Text)

Max Activating Token Index: 277

Text #15

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1749. Min Act: -0.1700

Data Index: 7164635 (Open Web Text)

Max Activating Token Index: 880

Text #16

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1368. Min Act: -0.1700

Data Index: 7277589 (Open Web Text)

Max Activating Token Index: 374

Text #17

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1410. Min Act: -0.1700

Data Index: 1813978 (Open Web Text)

Max Activating Token Index: 298

Text #18

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1814. Min Act: -0.1700

Data Index: 5638454 (Open Web Text)

Max Activating Token Index: 643

Text #19

Max Range: 4.5172. Min Range: -4.5172

Max Act: 4.1196. Min Act: -0.1700

Data Index: 5656815 (Open Web Text)

Max Activating Token Index: 879