Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 581 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.4402. Min Act: -0.1700

Data Index: 4150945 (Open Web Text)

Max Activating Token Index: 97

Text #1

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.4398. Min Act: -0.1700

Data Index: 5271292 (Open Web Text)

Max Activating Token Index: 342

Text #2

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3868. Min Act: -0.1700

Data Index: 2693335 (Open Web Text)

Max Activating Token Index: 640

Text #3

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3920. Min Act: -0.1700

Data Index: 2648359 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3779. Min Act: -0.1700

Data Index: 1900829 (Open Web Text)

Max Activating Token Index: 21

Text #5

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3516. Min Act: -0.1700

Data Index: 3107752 (Open Web Text)

Max Activating Token Index: 948

Text #6

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3384. Min Act: -0.1700

Data Index: 657826 (Open Web Text)

Max Activating Token Index: 808

Text #7

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3586. Min Act: -0.1700

Data Index: 3158647 (Open Web Text)

Max Activating Token Index: 893

Text #8

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3358. Min Act: -0.1700

Data Index: 5966520 (Open Web Text)

Max Activating Token Index: 733

Text #9

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3046. Min Act: -0.1700

Data Index: 5445395 (Open Web Text)

Max Activating Token Index: 770

Text #10

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3441. Min Act: -0.1700

Data Index: 4198360 (Open Web Text)

Max Activating Token Index: 920

Text #11

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3375. Min Act: -0.1700

Data Index: 6512811 (Open Web Text)

Max Activating Token Index: 948

Text #12

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3216. Min Act: -0.1700

Data Index: 8123292 (Open Web Text)

Max Activating Token Index: 702

Text #13

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3082. Min Act: -0.1700

Data Index: 6328827 (Open Web Text)

Max Activating Token Index: 596

Text #14

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3390. Min Act: -0.1700

Data Index: 964029 (Open Web Text)

Max Activating Token Index: 968

Text #15

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.2965. Min Act: -0.1700

Data Index: 833236 (Open Web Text)

Max Activating Token Index: 366

Text #16

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.2961. Min Act: -0.1700

Data Index: 964031 (Open Web Text)

Max Activating Token Index: 990

Text #17

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3070. Min Act: -0.1700

Data Index: 707025 (Open Web Text)

Max Activating Token Index: 18

Text #18

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.3135. Min Act: -0.1700

Data Index: 4341809 (Open Web Text)

Max Activating Token Index: 470

Text #19

Max Range: 3.4402. Min Range: -3.4402

Max Act: 3.2570. Min Act: -0.1700

Data Index: 3374432 (Open Web Text)

Max Activating Token Index: 183