Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 964 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.7489. Min Act: -0.1700

Data Index: 6839911 (Open Web Text)

Max Activating Token Index: 314

Text #1

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.5505. Min Act: -0.1700

Data Index: 3148867 (Open Web Text)

Max Activating Token Index: 124

Text #2

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4794. Min Act: -0.1700

Data Index: 3927591 (Open Web Text)

Max Activating Token Index: 584

Text #3

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4518. Min Act: -0.1700

Data Index: 8700185 (Open Web Text)

Max Activating Token Index: 63

Text #4

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4473. Min Act: -0.1700

Data Index: 6046749 (Open Web Text)

Max Activating Token Index: 917

Text #5

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4099. Min Act: -0.1700

Data Index: 4205371 (Open Web Text)

Max Activating Token Index: 363

Text #6

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4047. Min Act: -0.1700

Data Index: 2716364 (Open Web Text)

Max Activating Token Index: 279

Text #7

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4377. Min Act: -0.1700

Data Index: 1212682 (Open Web Text)

Max Activating Token Index: 427

Text #8

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4009. Min Act: -0.1700

Data Index: 2867183 (Open Web Text)

Max Activating Token Index: 915

Text #9

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3541. Min Act: -0.1700

Data Index: 31897 (Open Web Text)

Max Activating Token Index: 773

Text #10

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.4093. Min Act: -0.1700

Data Index: 2187272 (Open Web Text)

Max Activating Token Index: 609

Text #11

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3916. Min Act: -0.1700

Data Index: 5004919 (Open Web Text)

Max Activating Token Index: 510

Text #12

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3488. Min Act: -0.1700

Data Index: 5311142 (Open Web Text)

Max Activating Token Index: 623

Text #13

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.2987. Min Act: -0.1700

Data Index: 2718752 (Open Web Text)

Max Activating Token Index: 554

Text #14

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3331. Min Act: -0.1700

Data Index: 904351 (Open Web Text)

Max Activating Token Index: 412

Text #15

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3073. Min Act: -0.1700

Data Index: 1026364 (Open Web Text)

Max Activating Token Index: 759

Text #16

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3388. Min Act: -0.1700

Data Index: 7330959 (Open Web Text)

Max Activating Token Index: 972

Text #17

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.3070. Min Act: -0.1700

Data Index: 7755645 (Open Web Text)

Max Activating Token Index: 925

Text #18

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.2692. Min Act: -0.1700

Data Index: 3109591 (Open Web Text)

Max Activating Token Index: 447

Text #19

Max Range: 5.7489. Min Range: -5.7489

Max Act: 5.2429. Min Act: -0.1700

Data Index: 6519341 (Open Web Text)

Max Activating Token Index: 928