Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 851 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0929. Min Act: -0.1700

Data Index: 8321117 (Open Web Text)

Max Activating Token Index: 983

Text #1

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.1156. Min Act: -0.1700

Data Index: 7791432 (Open Web Text)

Max Activating Token Index: 896

Text #2

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.1608. Min Act: -0.1700

Data Index: 598011 (Open Web Text)

Max Activating Token Index: 559

Text #3

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0746. Min Act: -0.1700

Data Index: 373469 (Open Web Text)

Max Activating Token Index: 701

Text #4

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0660. Min Act: -0.1700

Data Index: 112332 (Open Web Text)

Max Activating Token Index: 162

Text #5

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0505. Min Act: -0.1700

Data Index: 1432229 (Open Web Text)

Max Activating Token Index: 723

Text #6

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0826. Min Act: -0.1700

Data Index: 2261183 (Open Web Text)

Max Activating Token Index: 573

Text #7

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0637. Min Act: -0.1700

Data Index: 7324033 (Open Web Text)

Max Activating Token Index: 957

Text #8

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0704. Min Act: -0.1700

Data Index: 1888065 (Open Web Text)

Max Activating Token Index: 960

Text #9

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0222. Min Act: -0.1700

Data Index: 995645 (Open Web Text)

Max Activating Token Index: 520

Text #10

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0217. Min Act: -0.1700

Data Index: 6920482 (Open Web Text)

Max Activating Token Index: 737

Text #11

Max Range: 4.1608. Min Range: -4.1608

Max Act: 3.9974. Min Act: -0.1700

Data Index: 3400390 (Open Web Text)

Max Activating Token Index: 964

Text #12

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0266. Min Act: -0.1700

Data Index: 7222503 (Open Web Text)

Max Activating Token Index: 519

Text #13

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0402. Min Act: -0.1700

Data Index: 4923359 (Open Web Text)

Max Activating Token Index: 966

Text #14

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0288. Min Act: -0.1700

Data Index: 4315105 (Open Web Text)

Max Activating Token Index: 847

Text #15

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0224. Min Act: -0.1700

Data Index: 3750014 (Open Web Text)

Max Activating Token Index: 105

Text #16

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0180. Min Act: -0.1700

Data Index: 1474596 (Open Web Text)

Max Activating Token Index: 548

Text #17

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0147. Min Act: -0.1700

Data Index: 2391319 (Open Web Text)

Max Activating Token Index: 901

Text #18

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0339. Min Act: -0.1700

Data Index: 8187010 (Open Web Text)

Max Activating Token Index: 994

Text #19

Max Range: 4.1608. Min Range: -4.1608

Max Act: 4.0237. Min Act: -0.1700

Data Index: 2644944 (Open Web Text)

Max Activating Token Index: 839