Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2597 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.9817. Min Act: -0.1700

Data Index: 8411201 (Open Web Text)

Max Activating Token Index: 466

Text #1

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.9124. Min Act: -0.1700

Data Index: 7602659 (Open Web Text)

Max Activating Token Index: 987

Text #2

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8897. Min Act: -0.1700

Data Index: 5787987 (Open Web Text)

Max Activating Token Index: 10

Text #3

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8900. Min Act: -0.1700

Data Index: 7334 (Open Web Text)

Max Activating Token Index: 269

Text #4

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8677. Min Act: -0.1700

Data Index: 1074049 (Open Web Text)

Max Activating Token Index: 604

Text #5

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8553. Min Act: -0.1700

Data Index: 2952321 (Open Web Text)

Max Activating Token Index: 387

Text #6

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8347. Min Act: -0.1700

Data Index: 5664186 (Open Web Text)

Max Activating Token Index: 286

Text #7

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8314. Min Act: -0.1700

Data Index: 8431245 (Open Web Text)

Max Activating Token Index: 886

Text #8

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8287. Min Act: -0.1700

Data Index: 2403244 (Open Web Text)

Max Activating Token Index: 763

Text #9

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8383. Min Act: -0.1700

Data Index: 4634836 (Open Web Text)

Max Activating Token Index: 440

Text #10

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8473. Min Act: -0.1700

Data Index: 5922016 (Open Web Text)

Max Activating Token Index: 177

Text #11

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8382. Min Act: -0.1700

Data Index: 274706 (Open Web Text)

Max Activating Token Index: 956

Text #12

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8615. Min Act: -0.1700

Data Index: 1522995 (Open Web Text)

Max Activating Token Index: 932

Text #13

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8316. Min Act: -0.1700

Data Index: 2123400 (Open Web Text)

Max Activating Token Index: 352

Text #14

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8318. Min Act: -0.1700

Data Index: 3984750 (Open Web Text)

Max Activating Token Index: 255

Text #15

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8125. Min Act: -0.1700

Data Index: 2574264 (Open Web Text)

Max Activating Token Index: 441

Text #16

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8280. Min Act: -0.1700

Data Index: 7183447 (Open Web Text)

Max Activating Token Index: 180

Text #17

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.7918. Min Act: -0.1700

Data Index: 5863630 (Open Web Text)

Max Activating Token Index: 557

Text #18

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.7966. Min Act: -0.1700

Data Index: 257263 (Open Web Text)

Max Activating Token Index: 169

Text #19

Max Range: 1.9817. Min Range: -1.9817

Max Act: 1.8075. Min Act: -0.1700

Data Index: 130984 (Open Web Text)

Max Activating Token Index: 88