Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 947 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1859. Min Act: -0.1700

Data Index: 2758116 (Open Web Text)

Max Activating Token Index: 964

Text #1

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1621. Min Act: -0.1700

Data Index: 6502864 (Open Web Text)

Max Activating Token Index: 299

Text #2

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1562. Min Act: -0.1700

Data Index: 7464903 (Open Web Text)

Max Activating Token Index: 209

Text #3

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1080. Min Act: -0.1700

Data Index: 8026551 (Open Web Text)

Max Activating Token Index: 24

Text #4

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1273. Min Act: -0.1700

Data Index: 3910863 (Open Web Text)

Max Activating Token Index: 512

Text #5

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1125. Min Act: -0.1700

Data Index: 5665134 (Open Web Text)

Max Activating Token Index: 596

Text #6

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.1307. Min Act: -0.1700

Data Index: 4830658 (Open Web Text)

Max Activating Token Index: 645

Text #7

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0884. Min Act: -0.1700

Data Index: 7464904 (Open Web Text)

Max Activating Token Index: 173

Text #8

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0876. Min Act: -0.1700

Data Index: 3032606 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0884. Min Act: -0.1700

Data Index: 2913168 (Open Web Text)

Max Activating Token Index: 970

Text #10

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0534. Min Act: -0.1700

Data Index: 1009045 (Open Web Text)

Max Activating Token Index: 1019

Text #11

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0702. Min Act: -0.1700

Data Index: 2450045 (Open Web Text)

Max Activating Token Index: 21

Text #12

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0563. Min Act: -0.1700

Data Index: 1428844 (Open Web Text)

Max Activating Token Index: 397

Text #13

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0338. Min Act: -0.1700

Data Index: 2883796 (Open Web Text)

Max Activating Token Index: 78

Text #14

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0707. Min Act: -0.1700

Data Index: 5417915 (Open Web Text)

Max Activating Token Index: 116

Text #15

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0458. Min Act: -0.1700

Data Index: 2492990 (Open Web Text)

Max Activating Token Index: 1009

Text #16

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0472. Min Act: -0.1700

Data Index: 5680322 (Open Web Text)

Max Activating Token Index: 8

Text #17

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0474. Min Act: -0.1700

Data Index: 4294038 (Open Web Text)

Max Activating Token Index: 610

Text #18

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0526. Min Act: -0.1700

Data Index: 7050878 (Open Web Text)

Max Activating Token Index: 568

Text #19

Max Range: 4.1859. Min Range: -4.1859

Max Act: 4.0344. Min Act: -0.1700

Data Index: 101628 (Open Web Text)

Max Activating Token Index: 634