Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1809 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.6192. Min Act: -0.1699

Data Index: 5955305 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.5815. Min Act: -0.1700

Data Index: 245649 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.5130. Min Act: -0.1700

Data Index: 709611 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.4994. Min Act: -0.1472

Data Index: 6555066 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.4945. Min Act: -0.1173

Data Index: 6555068 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3849. Min Act: -0.1695

Data Index: 7983511 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3502. Min Act: -0.1695

Data Index: 4375755 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3299. Min Act: -0.1661

Data Index: 7051930 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3154. Min Act: -0.1546

Data Index: 122383 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3145. Min Act: -0.1591

Data Index: 5102919 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3180. Min Act: -0.1449

Data Index: 4068695 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3195. Min Act: -0.1695

Data Index: 2444854 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.3132. Min Act: -0.1679

Data Index: 2443566 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2909. Min Act: -0.1681

Data Index: 4985681 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2878. Min Act: -0.1020

Data Index: 3959404 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2785. Min Act: -0.1189

Data Index: 3959407 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2456. Min Act: -0.1696

Data Index: 6284849 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2645. Min Act: -0.1698

Data Index: 7569117 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2550. Min Act: -0.1593

Data Index: 6438713 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 2.6192. Min Range: -2.6192

Max Act: 2.2390. Min Act: -0.1700

Data Index: 300685 (Open Web Text)

Max Activating Token Index: 1023