Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2862 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0909. Min Range: -5.0909

Max Act: 5.0909. Min Act: -0.1700

Data Index: 7560153 (Open Web Text)

Max Activating Token Index: 9

Text #1

Max Range: 5.0909. Min Range: -5.0909

Max Act: 5.0329. Min Act: -0.1700

Data Index: 1485367 (Open Web Text)

Max Activating Token Index: 760

Text #2

Max Range: 5.0909. Min Range: -5.0909

Max Act: 5.0399. Min Act: -0.1700

Data Index: 78414 (Open Web Text)

Max Activating Token Index: 391

Text #3

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9939. Min Act: -0.1700

Data Index: 1589003 (Open Web Text)

Max Activating Token Index: 690

Text #4

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9947. Min Act: -0.1700

Data Index: 5348433 (Open Web Text)

Max Activating Token Index: 300

Text #5

Max Range: 5.0909. Min Range: -5.0909

Max Act: 5.0000. Min Act: -0.1700

Data Index: 6753960 (Open Web Text)

Max Activating Token Index: 637

Text #6

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9508. Min Act: -0.1700

Data Index: 2653466 (Open Web Text)

Max Activating Token Index: 878

Text #7

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9568. Min Act: -0.1700

Data Index: 362364 (Open Web Text)

Max Activating Token Index: 33

Text #8

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9755. Min Act: -0.1700

Data Index: 7560154 (Open Web Text)

Max Activating Token Index: 584

Text #9

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9634. Min Act: -0.1700

Data Index: 7560152 (Open Web Text)

Max Activating Token Index: 780

Text #10

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9453. Min Act: -0.1700

Data Index: 2822455 (Open Web Text)

Max Activating Token Index: 653

Text #11

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9159. Min Act: -0.1700

Data Index: 3012404 (Open Web Text)

Max Activating Token Index: 76

Text #12

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9397. Min Act: -0.1700

Data Index: 1808073 (Open Web Text)

Max Activating Token Index: 885

Text #13

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9487. Min Act: -0.1700

Data Index: 3955506 (Open Web Text)

Max Activating Token Index: 288

Text #14

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9447. Min Act: -0.1700

Data Index: 8252963 (Open Web Text)

Max Activating Token Index: 946

Text #15

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.8946. Min Act: -0.1700

Data Index: 1364399 (Open Web Text)

Max Activating Token Index: 12

Text #16

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.8724. Min Act: -0.1700

Data Index: 6264583 (Open Web Text)

Max Activating Token Index: 813

Text #17

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9058. Min Act: -0.1700

Data Index: 6552366 (Open Web Text)

Max Activating Token Index: 626

Text #18

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.9011. Min Act: -0.1700

Data Index: 6517437 (Open Web Text)

Max Activating Token Index: 853

Text #19

Max Range: 5.0909. Min Range: -5.0909

Max Act: 4.8575. Min Act: -0.1700

Data Index: 5029663 (Open Web Text)

Max Activating Token Index: 180