Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2196 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.6733. Min Act: -0.1692

Data Index: 1837228 (Open Web Text)

Max Activating Token Index: 802

Text #1

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.6037. Min Act: -0.1658

Data Index: 1555127 (Open Web Text)

Max Activating Token Index: 730

Text #2

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.5283. Min Act: -0.1699

Data Index: 4132126 (Open Web Text)

Max Activating Token Index: 642

Text #3

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.5111. Min Act: -0.1695

Data Index: 471585 (Open Web Text)

Max Activating Token Index: 582

Text #4

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.5068. Min Act: -0.1694

Data Index: 2629002 (Open Web Text)

Max Activating Token Index: 382

Text #5

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.5223. Min Act: -0.1678

Data Index: 1634846 (Open Web Text)

Max Activating Token Index: 21

Text #6

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.5194. Min Act: -0.1626

Data Index: 8065819 (Open Web Text)

Max Activating Token Index: 543

Text #7

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4541. Min Act: -0.1699

Data Index: 7754214 (Open Web Text)

Max Activating Token Index: 866

Text #8

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.5016. Min Act: -0.1700

Data Index: 1346272 (Open Web Text)

Max Activating Token Index: 78

Text #9

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4971. Min Act: -0.1698

Data Index: 4573612 (Open Web Text)

Max Activating Token Index: 826

Text #10

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4138. Min Act: -0.1692

Data Index: 2100409 (Open Web Text)

Max Activating Token Index: 188

Text #11

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4377. Min Act: -0.1601

Data Index: 6932384 (Open Web Text)

Max Activating Token Index: 162

Text #12

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4267. Min Act: -0.1650

Data Index: 300658 (Open Web Text)

Max Activating Token Index: 701

Text #13

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4168. Min Act: -0.1693

Data Index: 5727701 (Open Web Text)

Max Activating Token Index: 702

Text #14

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4071. Min Act: -0.1691

Data Index: 2168728 (Open Web Text)

Max Activating Token Index: 283

Text #15

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4063. Min Act: -0.1699

Data Index: 1128142 (Open Web Text)

Max Activating Token Index: 120

Text #16

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.3965. Min Act: -0.1700

Data Index: 4791792 (Open Web Text)

Max Activating Token Index: 106

Text #17

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4072. Min Act: -0.1692

Data Index: 3710471 (Open Web Text)

Max Activating Token Index: 652

Text #18

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4293. Min Act: -0.1674

Data Index: 3616810 (Open Web Text)

Max Activating Token Index: 101

Text #19

Max Range: 3.6733. Min Range: -3.6733

Max Act: 3.4400. Min Act: -0.1694

Data Index: 3174570 (Open Web Text)

Max Activating Token Index: 238