Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1475 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.4958. Min Act: -0.1700

Data Index: 752932 (Open Web Text)

Max Activating Token Index: 200

Text #1

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.4623. Min Act: -0.1700

Data Index: 260865 (Open Web Text)

Max Activating Token Index: 146

Text #2

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.4191. Min Act: -0.1700

Data Index: 6754340 (Open Web Text)

Max Activating Token Index: 622

Text #3

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.4139. Min Act: -0.1700

Data Index: 6354044 (Open Web Text)

Max Activating Token Index: 78

Text #4

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3661. Min Act: -0.1700

Data Index: 6170005 (Open Web Text)

Max Activating Token Index: 476

Text #5

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3712. Min Act: -0.1700

Data Index: 7439805 (Open Web Text)

Max Activating Token Index: 210

Text #6

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3779. Min Act: -0.1700

Data Index: 5997780 (Open Web Text)

Max Activating Token Index: 507

Text #7

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3679. Min Act: -0.1700

Data Index: 1800613 (Open Web Text)

Max Activating Token Index: 960

Text #8

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3533. Min Act: -0.1700

Data Index: 7399046 (Open Web Text)

Max Activating Token Index: 587

Text #9

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3236. Min Act: -0.1700

Data Index: 2754219 (Open Web Text)

Max Activating Token Index: 889

Text #10

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3486. Min Act: -0.1700

Data Index: 5708834 (Open Web Text)

Max Activating Token Index: 596

Text #11

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3353. Min Act: -0.1700

Data Index: 7003880 (Open Web Text)

Max Activating Token Index: 580

Text #12

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3352. Min Act: -0.1700

Data Index: 4064153 (Open Web Text)

Max Activating Token Index: 748

Text #13

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3076. Min Act: -0.1700

Data Index: 287851 (Open Web Text)

Max Activating Token Index: 387

Text #14

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3177. Min Act: -0.1700

Data Index: 287853 (Open Web Text)

Max Activating Token Index: 931

Text #15

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3060. Min Act: -0.1700

Data Index: 7055825 (Open Web Text)

Max Activating Token Index: 356

Text #16

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.2990. Min Act: -0.1700

Data Index: 1674418 (Open Web Text)

Max Activating Token Index: 829

Text #17

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3343. Min Act: -0.1700

Data Index: 3608986 (Open Web Text)

Max Activating Token Index: 229

Text #18

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.2987. Min Act: -0.1700

Data Index: 3620530 (Open Web Text)

Max Activating Token Index: 751

Text #19

Max Range: 4.4958. Min Range: -4.4958

Max Act: 4.3151. Min Act: -0.1700

Data Index: 6407760 (Open Web Text)

Max Activating Token Index: 303