Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2396 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.7144. Min Act: -0.1699

Data Index: 1231427 (Open Web Text)

Max Activating Token Index: 700

Text #1

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.5630. Min Act: -0.1700

Data Index: 6079525 (Open Web Text)

Max Activating Token Index: 503

Text #2

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.4883. Min Act: -0.1700

Data Index: 6625823 (Open Web Text)

Max Activating Token Index: 92

Text #3

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.4383. Min Act: -0.1700

Data Index: 2538619 (Open Web Text)

Max Activating Token Index: 765

Text #4

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3908. Min Act: -0.1700

Data Index: 1659101 (Open Web Text)

Max Activating Token Index: 69

Text #5

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3821. Min Act: -0.1700

Data Index: 4717267 (Open Web Text)

Max Activating Token Index: 199

Text #6

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.4060. Min Act: -0.1700

Data Index: 7855583 (Open Web Text)

Max Activating Token Index: 855

Text #7

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.4104. Min Act: -0.1700

Data Index: 4663482 (Open Web Text)

Max Activating Token Index: 896

Text #8

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3854. Min Act: -0.1700

Data Index: 5661275 (Open Web Text)

Max Activating Token Index: 774

Text #9

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3773. Min Act: -0.1700

Data Index: 6992036 (Open Web Text)

Max Activating Token Index: 56

Text #10

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3641. Min Act: -0.1700

Data Index: 782215 (Open Web Text)

Max Activating Token Index: 610

Text #11

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3332. Min Act: -0.1700

Data Index: 2181845 (Open Web Text)

Max Activating Token Index: 399

Text #12

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3438. Min Act: -0.1700

Data Index: 3723773 (Open Web Text)

Max Activating Token Index: 731

Text #13

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3087. Min Act: -0.1700

Data Index: 2440630 (Open Web Text)

Max Activating Token Index: 395

Text #14

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.2766. Min Act: -0.1700

Data Index: 3162930 (Open Web Text)

Max Activating Token Index: 316

Text #15

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3097. Min Act: -0.1700

Data Index: 5618188 (Open Web Text)

Max Activating Token Index: 452

Text #16

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3001. Min Act: -0.1700

Data Index: 2360619 (Open Web Text)

Max Activating Token Index: 453

Text #17

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3112. Min Act: -0.1700

Data Index: 5293263 (Open Web Text)

Max Activating Token Index: 532

Text #18

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.3077. Min Act: -0.1700

Data Index: 3453665 (Open Web Text)

Max Activating Token Index: 160

Text #19

Max Range: 4.7144. Min Range: -4.7144

Max Act: 4.2880. Min Act: -0.1700

Data Index: 5661272 (Open Web Text)

Max Activating Token Index: 422