Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 0 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2773. Min Range: -5.2773

Max Act: 5.2773. Min Act: -0.1687

Data Index: 7696981 (Open Web Text)

Max Activating Token Index: 139

Text #1

Max Range: 5.2773. Min Range: -5.2773

Max Act: 5.1177. Min Act: -0.1700

Data Index: 6990001 (Open Web Text)

Max Activating Token Index: 102

Text #2

Max Range: 5.2773. Min Range: -5.2773

Max Act: 5.0415. Min Act: -0.1690

Data Index: 4188587 (Open Web Text)

Max Activating Token Index: 183

Text #3

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.9367. Min Act: -0.1623

Data Index: 6491016 (Open Web Text)

Max Activating Token Index: 467

Text #4

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.9272. Min Act: -0.1700

Data Index: 4663773 (Open Web Text)

Max Activating Token Index: 159

Text #5

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.9341. Min Act: -0.1700

Data Index: 8470531 (Open Web Text)

Max Activating Token Index: 131

Text #6

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.8687. Min Act: -0.1700

Data Index: 8445350 (Open Web Text)

Max Activating Token Index: 562

Text #7

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.8752. Min Act: -0.1529

Data Index: 5277658 (Open Web Text)

Max Activating Token Index: 791

Text #8

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.8766. Min Act: -0.1700

Data Index: 1033483 (Open Web Text)

Max Activating Token Index: 102

Text #9

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.8544. Min Act: -0.1699

Data Index: 6510830 (Open Web Text)

Max Activating Token Index: 634

Text #10

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.8363. Min Act: -0.1353

Data Index: 1776957 (Open Web Text)

Max Activating Token Index: 73

Text #11

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7882. Min Act: -0.1235

Data Index: 6366449 (Open Web Text)

Max Activating Token Index: 125

Text #12

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7551. Min Act: -0.1700

Data Index: 1683420 (Open Web Text)

Max Activating Token Index: 602

Text #13

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7629. Min Act: -0.1621

Data Index: 6954490 (Open Web Text)

Max Activating Token Index: 519

Text #14

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7781. Min Act: -0.1593

Data Index: 292909 (Open Web Text)

Max Activating Token Index: 134

Text #15

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7849. Min Act: -0.1697

Data Index: 4786432 (Open Web Text)

Max Activating Token Index: 128

Text #16

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7610. Min Act: -0.1580

Data Index: 2920066 (Open Web Text)

Max Activating Token Index: 122

Text #17

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7674. Min Act: -0.1695

Data Index: 8092337 (Open Web Text)

Max Activating Token Index: 951

Text #18

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7048. Min Act: -0.0956

Data Index: 7235282 (Open Web Text)

Max Activating Token Index: 171

Text #19

Max Range: 5.2773. Min Range: -5.2773

Max Act: 4.7421. Min Act: -0.1607

Data Index: 2135199 (Open Web Text)

Max Activating Token Index: 119