Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1025 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.7323. Min Act: -0.1700

Data Index: 4320071 (Open Web Text)

Max Activating Token Index: 661

Text #1

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.7163. Min Act: -0.1700

Data Index: 1982065 (Open Web Text)

Max Activating Token Index: 307

Text #2

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.5386. Min Act: -0.1700

Data Index: 3222841 (Open Web Text)

Max Activating Token Index: 931

Text #3

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.5108. Min Act: -0.1700

Data Index: 2259035 (Open Web Text)

Max Activating Token Index: 1000

Text #4

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4919. Min Act: -0.1700

Data Index: 430752 (Open Web Text)

Max Activating Token Index: 455

Text #5

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4970. Min Act: -0.1700

Data Index: 2067540 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4794. Min Act: -0.1700

Data Index: 3061000 (Open Web Text)

Max Activating Token Index: 352

Text #7

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4520. Min Act: -0.1700

Data Index: 2425176 (Open Web Text)

Max Activating Token Index: 597

Text #8

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4553. Min Act: -0.1700

Data Index: 3408025 (Open Web Text)

Max Activating Token Index: 788

Text #9

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4431. Min Act: -0.1700

Data Index: 8589909 (Open Web Text)

Max Activating Token Index: 972

Text #10

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4666. Min Act: -0.1700

Data Index: 1374626 (Open Web Text)

Max Activating Token Index: 881

Text #11

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3913. Min Act: -0.1700

Data Index: 8355396 (Open Web Text)

Max Activating Token Index: 79

Text #12

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.4053. Min Act: -0.1700

Data Index: 5407368 (Open Web Text)

Max Activating Token Index: 663

Text #13

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3884. Min Act: -0.1700

Data Index: 8704481 (Open Web Text)

Max Activating Token Index: 761

Text #14

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3982. Min Act: -0.1700

Data Index: 1794147 (Open Web Text)

Max Activating Token Index: 646

Text #15

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3707. Min Act: -0.1700

Data Index: 7665030 (Open Web Text)

Max Activating Token Index: 1010

Text #16

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3899. Min Act: -0.1700

Data Index: 8533219 (Open Web Text)

Max Activating Token Index: 211

Text #17

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3920. Min Act: -0.1700

Data Index: 1366704 (Open Web Text)

Max Activating Token Index: 709

Text #18

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3543. Min Act: -0.1700

Data Index: 884186 (Open Web Text)

Max Activating Token Index: 59

Text #19

Max Range: 3.7323. Min Range: -3.7323

Max Act: 3.3633. Min Act: -0.1700

Data Index: 8316992 (Open Web Text)

Max Activating Token Index: 926