Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2757 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.6757. Min Act: -0.1700

Data Index: 5719829 (Open Web Text)

Max Activating Token Index: 359

Text #1

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.6512. Min Act: -0.1700

Data Index: 4297485 (Open Web Text)

Max Activating Token Index: 276

Text #2

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.6310. Min Act: -0.1700

Data Index: 7330653 (Open Web Text)

Max Activating Token Index: 468

Text #3

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5949. Min Act: -0.1700

Data Index: 3069238 (Open Web Text)

Max Activating Token Index: 307

Text #4

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5956. Min Act: -0.1700

Data Index: 1297615 (Open Web Text)

Max Activating Token Index: 605

Text #5

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5928. Min Act: -0.1700

Data Index: 6858792 (Open Web Text)

Max Activating Token Index: 306

Text #6

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.6260. Min Act: -0.1700

Data Index: 5229257 (Open Web Text)

Max Activating Token Index: 570

Text #7

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5927. Min Act: -0.1700

Data Index: 2401937 (Open Web Text)

Max Activating Token Index: 33

Text #8

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.6070. Min Act: -0.1700

Data Index: 5575061 (Open Web Text)

Max Activating Token Index: 174

Text #9

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5783. Min Act: -0.1700

Data Index: 5643127 (Open Web Text)

Max Activating Token Index: 475

Text #10

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5874. Min Act: -0.1700

Data Index: 879730 (Open Web Text)

Max Activating Token Index: 256

Text #11

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5769. Min Act: -0.1700

Data Index: 4302096 (Open Web Text)

Max Activating Token Index: 138

Text #12

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5660. Min Act: -0.1700

Data Index: 2853532 (Open Web Text)

Max Activating Token Index: 221

Text #13

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5716. Min Act: -0.1700

Data Index: 8518373 (Open Web Text)

Max Activating Token Index: 192

Text #14

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5704. Min Act: -0.1700

Data Index: 7795324 (Open Web Text)

Max Activating Token Index: 439

Text #15

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5820. Min Act: -0.1700

Data Index: 7257866 (Open Web Text)

Max Activating Token Index: 75

Text #16

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5716. Min Act: -0.1700

Data Index: 5335199 (Open Web Text)

Max Activating Token Index: 878

Text #17

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5706. Min Act: -0.1700

Data Index: 5175651 (Open Web Text)

Max Activating Token Index: 41

Text #18

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5604. Min Act: -0.1700

Data Index: 422782 (Open Web Text)

Max Activating Token Index: 627

Text #19

Max Range: 4.6757. Min Range: -4.6757

Max Act: 4.5347. Min Act: -0.1700

Data Index: 1100425 (Open Web Text)

Max Activating Token Index: 250