Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2415 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.5467. Min Act: -0.1700

Data Index: 6911456 (Open Web Text)

Max Activating Token Index: 123

Text #1

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.5025. Min Act: -0.1700

Data Index: 4287957 (Open Web Text)

Max Activating Token Index: 473

Text #2

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.4786. Min Act: -0.1700

Data Index: 6931025 (Open Web Text)

Max Activating Token Index: 253

Text #3

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.4564. Min Act: -0.1700

Data Index: 3848117 (Open Web Text)

Max Activating Token Index: 627

Text #4

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.4120. Min Act: -0.1700

Data Index: 4308274 (Open Web Text)

Max Activating Token Index: 438

Text #5

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.4109. Min Act: -0.1700

Data Index: 1903402 (Open Web Text)

Max Activating Token Index: 865

Text #6

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.4007. Min Act: -0.1700

Data Index: 5630319 (Open Web Text)

Max Activating Token Index: 61

Text #7

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3912. Min Act: -0.1700

Data Index: 3801016 (Open Web Text)

Max Activating Token Index: 740

Text #8

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3728. Min Act: -0.1700

Data Index: 2081715 (Open Web Text)

Max Activating Token Index: 542

Text #9

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3684. Min Act: -0.1700

Data Index: 7408257 (Open Web Text)

Max Activating Token Index: 242

Text #10

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3595. Min Act: -0.1700

Data Index: 4645859 (Open Web Text)

Max Activating Token Index: 285

Text #11

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3601. Min Act: -0.1700

Data Index: 6400786 (Open Web Text)

Max Activating Token Index: 615

Text #12

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3527. Min Act: -0.1700

Data Index: 2108599 (Open Web Text)

Max Activating Token Index: 926

Text #13

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3545. Min Act: -0.1700

Data Index: 5648492 (Open Web Text)

Max Activating Token Index: 869

Text #14

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3364. Min Act: -0.1700

Data Index: 1908811 (Open Web Text)

Max Activating Token Index: 511

Text #15

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3330. Min Act: -0.1700

Data Index: 7588402 (Open Web Text)

Max Activating Token Index: 913

Text #16

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3251. Min Act: -0.1700

Data Index: 5480243 (Open Web Text)

Max Activating Token Index: 944

Text #17

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3221. Min Act: -0.1700

Data Index: 2370353 (Open Web Text)

Max Activating Token Index: 468

Text #18

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3249. Min Act: -0.1700

Data Index: 6930972 (Open Web Text)

Max Activating Token Index: 375

Text #19

Max Range: 3.5467. Min Range: -3.5467

Max Act: 3.3057. Min Act: -0.1700

Data Index: 1980496 (Open Web Text)

Max Activating Token Index: 243