Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2141 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.4634. Min Act: -0.1700

Data Index: 7327615 (Open Web Text)

Max Activating Token Index: 950

Text #1

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.4538. Min Act: -0.1700

Data Index: 3288017 (Open Web Text)

Max Activating Token Index: 451

Text #2

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.4135. Min Act: -0.1700

Data Index: 2835717 (Open Web Text)

Max Activating Token Index: 955

Text #3

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.4196. Min Act: -0.1700

Data Index: 7548740 (Open Web Text)

Max Activating Token Index: 702

Text #4

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3980. Min Act: -0.1700

Data Index: 6474828 (Open Web Text)

Max Activating Token Index: 808

Text #5

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.4084. Min Act: -0.1700

Data Index: 5814425 (Open Web Text)

Max Activating Token Index: 122

Text #6

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3989. Min Act: -0.1700

Data Index: 5814409 (Open Web Text)

Max Activating Token Index: 680

Text #7

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3674. Min Act: -0.1700

Data Index: 5799162 (Open Web Text)

Max Activating Token Index: 272

Text #8

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3805. Min Act: -0.1700

Data Index: 5814413 (Open Web Text)

Max Activating Token Index: 170

Text #9

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3923. Min Act: -0.1700

Data Index: 5367531 (Open Web Text)

Max Activating Token Index: 751

Text #10

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3755. Min Act: -0.1700

Data Index: 2243335 (Open Web Text)

Max Activating Token Index: 917

Text #11

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3397. Min Act: -0.1700

Data Index: 250442 (Open Web Text)

Max Activating Token Index: 643

Text #12

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3400. Min Act: -0.1700

Data Index: 5814414 (Open Web Text)

Max Activating Token Index: 127

Text #13

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3251. Min Act: -0.1700

Data Index: 265587 (Open Web Text)

Max Activating Token Index: 901

Text #14

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3298. Min Act: -0.1700

Data Index: 5407082 (Open Web Text)

Max Activating Token Index: 874

Text #15

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3242. Min Act: -0.1700

Data Index: 5876778 (Open Web Text)

Max Activating Token Index: 179

Text #16

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3421. Min Act: -0.1700

Data Index: 5367537 (Open Web Text)

Max Activating Token Index: 84

Text #17

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3389. Min Act: -0.1700

Data Index: 2982229 (Open Web Text)

Max Activating Token Index: 569

Text #18

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3474. Min Act: -0.1700

Data Index: 6152712 (Open Web Text)

Max Activating Token Index: 917

Text #19

Max Range: 4.4634. Min Range: -4.4634

Max Act: 4.3390. Min Act: -0.1700

Data Index: 4362064 (Open Web Text)

Max Activating Token Index: 618