Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 354 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.3646. Min Act: -0.1700

Data Index: 3571650 (Open Web Text)

Max Activating Token Index: 332

Text #1

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.3541. Min Act: -0.1700

Data Index: 3951242 (Open Web Text)

Max Activating Token Index: 307

Text #2

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.2596. Min Act: -0.1700

Data Index: 6642432 (Open Web Text)

Max Activating Token Index: 324

Text #3

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.2865. Min Act: -0.1700

Data Index: 3916624 (Open Web Text)

Max Activating Token Index: 67

Text #4

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.2522. Min Act: -0.1700

Data Index: 6619993 (Open Web Text)

Max Activating Token Index: 663

Text #5

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1974. Min Act: -0.1700

Data Index: 6607925 (Open Web Text)

Max Activating Token Index: 553

Text #6

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.2348. Min Act: -0.1700

Data Index: 3036064 (Open Web Text)

Max Activating Token Index: 82

Text #7

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.2127. Min Act: -0.1700

Data Index: 5615783 (Open Web Text)

Max Activating Token Index: 93

Text #8

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1941. Min Act: -0.1700

Data Index: 4116450 (Open Web Text)

Max Activating Token Index: 244

Text #9

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1841. Min Act: -0.1700

Data Index: 7635622 (Open Web Text)

Max Activating Token Index: 228

Text #10

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1705. Min Act: -0.1700

Data Index: 2271243 (Open Web Text)

Max Activating Token Index: 985

Text #11

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1697. Min Act: -0.1700

Data Index: 7298314 (Open Web Text)

Max Activating Token Index: 514

Text #12

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1989. Min Act: -0.1700

Data Index: 6736169 (Open Web Text)

Max Activating Token Index: 379

Text #13

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1549. Min Act: -0.1700

Data Index: 1236586 (Open Web Text)

Max Activating Token Index: 254

Text #14

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1454. Min Act: -0.1700

Data Index: 8253523 (Open Web Text)

Max Activating Token Index: 246

Text #15

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1525. Min Act: -0.1700

Data Index: 172083 (Open Web Text)

Max Activating Token Index: 295

Text #16

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1434. Min Act: -0.1700

Data Index: 2406101 (Open Web Text)

Max Activating Token Index: 245

Text #17

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1220. Min Act: -0.1700

Data Index: 4353260 (Open Web Text)

Max Activating Token Index: 477

Text #18

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1379. Min Act: -0.1700

Data Index: 4639458 (Open Web Text)

Max Activating Token Index: 51

Text #19

Max Range: 5.3646. Min Range: -5.3646

Max Act: 5.1300. Min Act: -0.1700

Data Index: 5949208 (Open Web Text)

Max Activating Token Index: 208