Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1631 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4615. Min Act: -0.1700

Data Index: 4243267 (Open Web Text)

Max Activating Token Index: 519

Text #1

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4555. Min Act: -0.1700

Data Index: 7851203 (Open Web Text)

Max Activating Token Index: 642

Text #2

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4669. Min Act: -0.1700

Data Index: 3786676 (Open Web Text)

Max Activating Token Index: 464

Text #3

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4378. Min Act: -0.1700

Data Index: 5451932 (Open Web Text)

Max Activating Token Index: 489

Text #4

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4492. Min Act: -0.1700

Data Index: 4571541 (Open Web Text)

Max Activating Token Index: 132

Text #5

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4231. Min Act: -0.1700

Data Index: 6783218 (Open Web Text)

Max Activating Token Index: 494

Text #6

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4492. Min Act: -0.1700

Data Index: 853594 (Open Web Text)

Max Activating Token Index: 705

Text #7

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4176. Min Act: -0.1700

Data Index: 6059538 (Open Web Text)

Max Activating Token Index: 960

Text #8

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.4014. Min Act: -0.1700

Data Index: 7306828 (Open Web Text)

Max Activating Token Index: 829

Text #9

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3962. Min Act: -0.1700

Data Index: 3435884 (Open Web Text)

Max Activating Token Index: 938

Text #10

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3988. Min Act: -0.1700

Data Index: 4839761 (Open Web Text)

Max Activating Token Index: 116

Text #11

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3815. Min Act: -0.1700

Data Index: 2765460 (Open Web Text)

Max Activating Token Index: 480

Text #12

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3956. Min Act: -0.1700

Data Index: 4388347 (Open Web Text)

Max Activating Token Index: 328

Text #13

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3770. Min Act: -0.1700

Data Index: 72711 (Open Web Text)

Max Activating Token Index: 732

Text #14

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3659. Min Act: -0.1700

Data Index: 187744 (Open Web Text)

Max Activating Token Index: 163

Text #15

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3650. Min Act: -0.1700

Data Index: 2239284 (Open Web Text)

Max Activating Token Index: 754

Text #16

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3439. Min Act: -0.1700

Data Index: 2340020 (Open Web Text)

Max Activating Token Index: 270

Text #17

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3541. Min Act: -0.1700

Data Index: 7001001 (Open Web Text)

Max Activating Token Index: 346

Text #18

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3349. Min Act: -0.1700

Data Index: 2858002 (Open Web Text)

Max Activating Token Index: 311

Text #19

Max Range: 3.4669. Min Range: -3.4669

Max Act: 3.3663. Min Act: -0.1700

Data Index: 7740879 (Open Web Text)

Max Activating Token Index: 763