Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 507 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.6565. Min Act: -0.1700

Data Index: 4528139 (Open Web Text)

Max Activating Token Index: 891

Text #1

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.4550. Min Act: -0.1700

Data Index: 4695280 (Open Web Text)

Max Activating Token Index: 437

Text #2

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.4506. Min Act: -0.1700

Data Index: 2678090 (Open Web Text)

Max Activating Token Index: 500

Text #3

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.4334. Min Act: -0.1700

Data Index: 3538207 (Open Web Text)

Max Activating Token Index: 283

Text #4

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.4278. Min Act: -0.1700

Data Index: 1372471 (Open Web Text)

Max Activating Token Index: 773

Text #5

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3932. Min Act: -0.1700

Data Index: 2021659 (Open Web Text)

Max Activating Token Index: 461

Text #6

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.4038. Min Act: -0.1700

Data Index: 7008194 (Open Web Text)

Max Activating Token Index: 992

Text #7

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.4015. Min Act: -0.1700

Data Index: 4043760 (Open Web Text)

Max Activating Token Index: 746

Text #8

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3934. Min Act: -0.1700

Data Index: 7008195 (Open Web Text)

Max Activating Token Index: 43

Text #9

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3560. Min Act: -0.1700

Data Index: 3972296 (Open Web Text)

Max Activating Token Index: 536

Text #10

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3536. Min Act: -0.1700

Data Index: 2701047 (Open Web Text)

Max Activating Token Index: 520

Text #11

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3447. Min Act: -0.1700

Data Index: 5733847 (Open Web Text)

Max Activating Token Index: 62

Text #12

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3736. Min Act: -0.1700

Data Index: 7262922 (Open Web Text)

Max Activating Token Index: 277

Text #13

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3296. Min Act: -0.1700

Data Index: 989515 (Open Web Text)

Max Activating Token Index: 494

Text #14

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3955. Min Act: -0.1700

Data Index: 6934263 (Open Web Text)

Max Activating Token Index: 291

Text #15

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.2933. Min Act: -0.1700

Data Index: 2371204 (Open Web Text)

Max Activating Token Index: 913

Text #16

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3060. Min Act: -0.1700

Data Index: 5456335 (Open Web Text)

Max Activating Token Index: 597

Text #17

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3619. Min Act: -0.1700

Data Index: 8648402 (Open Web Text)

Max Activating Token Index: 677

Text #18

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3574. Min Act: -0.1700

Data Index: 917260 (Open Web Text)

Max Activating Token Index: 384

Text #19

Max Range: 3.6565. Min Range: -3.6565

Max Act: 3.3479. Min Act: -0.1700

Data Index: 2293042 (Open Web Text)

Max Activating Token Index: 988