Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1335 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.1279. Min Act: -0.1700

Data Index: 7680590 (Open Web Text)

Max Activating Token Index: 798

Text #1

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.1049. Min Act: -0.1700

Data Index: 5260873 (Open Web Text)

Max Activating Token Index: 734

Text #2

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0857. Min Act: -0.1699

Data Index: 1237258 (Open Web Text)

Max Activating Token Index: 463

Text #3

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0720. Min Act: -0.1621

Data Index: 913065 (Open Web Text)

Max Activating Token Index: 194

Text #4

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0657. Min Act: -0.1700

Data Index: 5027858 (Open Web Text)

Max Activating Token Index: 801

Text #5

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0500. Min Act: -0.1696

Data Index: 5879376 (Open Web Text)

Max Activating Token Index: 560

Text #6

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0510. Min Act: -0.1700

Data Index: 8766557 (Open Web Text)

Max Activating Token Index: 296

Text #7

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0318. Min Act: -0.1700

Data Index: 1717015 (Open Web Text)

Max Activating Token Index: 219

Text #8

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0318. Min Act: -0.1700

Data Index: 8247535 (Open Web Text)

Max Activating Token Index: 129

Text #9

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0350. Min Act: -0.1700

Data Index: 3115103 (Open Web Text)

Max Activating Token Index: 859

Text #10

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0204. Min Act: -0.1700

Data Index: 913064 (Open Web Text)

Max Activating Token Index: 860

Text #11

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0208. Min Act: -0.1700

Data Index: 2041643 (Open Web Text)

Max Activating Token Index: 85

Text #12

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0002. Min Act: -0.1700

Data Index: 3896729 (Open Web Text)

Max Activating Token Index: 490

Text #13

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0268. Min Act: -0.1700

Data Index: 4035363 (Open Web Text)

Max Activating Token Index: 728

Text #14

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0258. Min Act: -0.1700

Data Index: 829076 (Open Web Text)

Max Activating Token Index: 497

Text #15

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0013. Min Act: -0.1700

Data Index: 6238042 (Open Web Text)

Max Activating Token Index: 825

Text #16

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0017. Min Act: -0.1700

Data Index: 2996781 (Open Web Text)

Max Activating Token Index: 556

Text #17

Max Range: 2.1279. Min Range: -2.1279

Max Act: 1.9904. Min Act: -0.1700

Data Index: 7841931 (Open Web Text)

Max Activating Token Index: 759

Text #18

Max Range: 2.1279. Min Range: -2.1279

Max Act: 2.0002. Min Act: -0.1632

Data Index: 3863741 (Open Web Text)

Max Activating Token Index: 734

Text #19

Max Range: 2.1279. Min Range: -2.1279

Max Act: 1.9979. Min Act: -0.1700

Data Index: 3901598 (Open Web Text)

Max Activating Token Index: 1006