Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2765 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.4395. Min Act: -0.1700

Data Index: 634977 (Open Web Text)

Max Activating Token Index: 958

Text #1

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.4293. Min Act: -0.1700

Data Index: 5202829 (Open Web Text)

Max Activating Token Index: 112

Text #2

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.4150. Min Act: -0.1700

Data Index: 2236 (Open Web Text)

Max Activating Token Index: 976

Text #3

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.4008. Min Act: -0.1700

Data Index: 1888581 (Open Web Text)

Max Activating Token Index: 243

Text #4

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3957. Min Act: -0.1700

Data Index: 4052103 (Open Web Text)

Max Activating Token Index: 97

Text #5

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3844. Min Act: -0.1700

Data Index: 8276314 (Open Web Text)

Max Activating Token Index: 21

Text #6

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3769. Min Act: -0.1700

Data Index: 5473726 (Open Web Text)

Max Activating Token Index: 624

Text #7

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3710. Min Act: -0.1700

Data Index: 5779871 (Open Web Text)

Max Activating Token Index: 781

Text #8

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3681. Min Act: -0.1700

Data Index: 8760090 (Open Web Text)

Max Activating Token Index: 812

Text #9

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3637. Min Act: -0.1700

Data Index: 7000318 (Open Web Text)

Max Activating Token Index: 420

Text #10

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3787. Min Act: -0.1700

Data Index: 3288285 (Open Web Text)

Max Activating Token Index: 943

Text #11

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3605. Min Act: -0.1700

Data Index: 4256265 (Open Web Text)

Max Activating Token Index: 49

Text #12

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3646. Min Act: -0.1700

Data Index: 5087811 (Open Web Text)

Max Activating Token Index: 774

Text #13

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3504. Min Act: -0.1700

Data Index: 5409570 (Open Web Text)

Max Activating Token Index: 733

Text #14

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3619. Min Act: -0.1700

Data Index: 3965211 (Open Web Text)

Max Activating Token Index: 950

Text #15

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3511. Min Act: -0.1700

Data Index: 7079239 (Open Web Text)

Max Activating Token Index: 389

Text #16

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3497. Min Act: -0.1700

Data Index: 5006985 (Open Web Text)

Max Activating Token Index: 456

Text #17

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3596. Min Act: -0.1700

Data Index: 1545719 (Open Web Text)

Max Activating Token Index: 409

Text #18

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3546. Min Act: -0.1700

Data Index: 323109 (Open Web Text)

Max Activating Token Index: 940

Text #19

Max Range: 0.4395. Min Range: -0.4395

Max Act: 0.3551. Min Act: -0.1700

Data Index: 2304505 (Open Web Text)

Max Activating Token Index: 81