Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 655 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1795. Min Act: -0.1700

Data Index: 812205 (Open Web Text)

Max Activating Token Index: 316

Text #1

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1894. Min Act: -0.1700

Data Index: 4566998 (Open Web Text)

Max Activating Token Index: 276

Text #2

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1740. Min Act: -0.1700

Data Index: 6129034 (Open Web Text)

Max Activating Token Index: 56

Text #3

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1322. Min Act: -0.1700

Data Index: 2150765 (Open Web Text)

Max Activating Token Index: 120

Text #4

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1516. Min Act: -0.1700

Data Index: 5872066 (Open Web Text)

Max Activating Token Index: 523

Text #5

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1369. Min Act: -0.1700

Data Index: 6240431 (Open Web Text)

Max Activating Token Index: 410

Text #6

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1121. Min Act: -0.1700

Data Index: 4164999 (Open Web Text)

Max Activating Token Index: 297

Text #7

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.1004. Min Act: -0.1700

Data Index: 6153781 (Open Web Text)

Max Activating Token Index: 677

Text #8

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0727. Min Act: -0.1700

Data Index: 3734807 (Open Web Text)

Max Activating Token Index: 455

Text #9

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0896. Min Act: -0.1700

Data Index: 4027663 (Open Web Text)

Max Activating Token Index: 618

Text #10

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0643. Min Act: -0.1700

Data Index: 3079486 (Open Web Text)

Max Activating Token Index: 336

Text #11

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0544. Min Act: -0.1700

Data Index: 2835784 (Open Web Text)

Max Activating Token Index: 863

Text #12

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0408. Min Act: -0.1700

Data Index: 1612013 (Open Web Text)

Max Activating Token Index: 592

Text #13

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0721. Min Act: -0.1700

Data Index: 1746194 (Open Web Text)

Max Activating Token Index: 583

Text #14

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0372. Min Act: -0.1700

Data Index: 277629 (Open Web Text)

Max Activating Token Index: 402

Text #15

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0414. Min Act: -0.1700

Data Index: 1636920 (Open Web Text)

Max Activating Token Index: 338

Text #16

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0269. Min Act: -0.1700

Data Index: 8208150 (Open Web Text)

Max Activating Token Index: 407

Text #17

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0534. Min Act: -0.1700

Data Index: 7257682 (Open Web Text)

Max Activating Token Index: 572

Text #18

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0080. Min Act: -0.1700

Data Index: 1215811 (Open Web Text)

Max Activating Token Index: 684

Text #19

Max Range: 2.1894. Min Range: -2.1894

Max Act: 2.0172. Min Act: -0.1700

Data Index: 878425 (Open Web Text)

Max Activating Token Index: 206