Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1354 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.5354. Min Act: -0.1700

Data Index: 6555238 (Open Web Text)

Max Activating Token Index: 686

Text #1

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.4244. Min Act: -0.1700

Data Index: 7113811 (Open Web Text)

Max Activating Token Index: 451

Text #2

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.4195. Min Act: -0.1700

Data Index: 4251371 (Open Web Text)

Max Activating Token Index: 489

Text #3

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.4031. Min Act: -0.1700

Data Index: 2141701 (Open Web Text)

Max Activating Token Index: 991

Text #4

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.4471. Min Act: -0.1700

Data Index: 1368226 (Open Web Text)

Max Activating Token Index: 472

Text #5

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.4161. Min Act: -0.1700

Data Index: 3415023 (Open Web Text)

Max Activating Token Index: 33

Text #6

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.3861. Min Act: -0.1700

Data Index: 578503 (Open Web Text)

Max Activating Token Index: 660

Text #7

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.4010. Min Act: -0.1700

Data Index: 4251383 (Open Web Text)

Max Activating Token Index: 119

Text #8

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.3318. Min Act: -0.1700

Data Index: 2958295 (Open Web Text)

Max Activating Token Index: 304

Text #9

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.3130. Min Act: -0.1700

Data Index: 2570483 (Open Web Text)

Max Activating Token Index: 354

Text #10

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.3449. Min Act: -0.1700

Data Index: 7084999 (Open Web Text)

Max Activating Token Index: 225

Text #11

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.3774. Min Act: -0.1700

Data Index: 8323048 (Open Web Text)

Max Activating Token Index: 562

Text #12

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2937. Min Act: -0.1700

Data Index: 6750479 (Open Web Text)

Max Activating Token Index: 387

Text #13

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2905. Min Act: -0.1700

Data Index: 4766950 (Open Web Text)

Max Activating Token Index: 1014

Text #14

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2934. Min Act: -0.1700

Data Index: 2092409 (Open Web Text)

Max Activating Token Index: 342

Text #15

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2622. Min Act: -0.1700

Data Index: 1029177 (Open Web Text)

Max Activating Token Index: 196

Text #16

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2870. Min Act: -0.1700

Data Index: 2307959 (Open Web Text)

Max Activating Token Index: 231

Text #17

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2910. Min Act: -0.1700

Data Index: 5556784 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2959. Min Act: -0.1700

Data Index: 6481248 (Open Web Text)

Max Activating Token Index: 418

Text #19

Max Range: 4.5354. Min Range: -4.5354

Max Act: 4.2655. Min Act: -0.1700

Data Index: 3128844 (Open Web Text)

Max Activating Token Index: 52