Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1267 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3695. Min Act: -0.1640

Data Index: 143979 (Open Web Text)

Max Activating Token Index: 946

Text #1

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3495. Min Act: -0.1640

Data Index: 4662112 (Open Web Text)

Max Activating Token Index: 968

Text #2

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3381. Min Act: -0.1640

Data Index: 8685439 (Open Web Text)

Max Activating Token Index: 746

Text #3

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3396. Min Act: -0.1640

Data Index: 30208 (Open Web Text)

Max Activating Token Index: 961

Text #4

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3339. Min Act: -0.1640

Data Index: 4421637 (Open Web Text)

Max Activating Token Index: 936

Text #5

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3304. Min Act: -0.1640

Data Index: 4130651 (Open Web Text)

Max Activating Token Index: 847

Text #6

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3266. Min Act: -0.1640

Data Index: 7358872 (Open Web Text)

Max Activating Token Index: 833

Text #7

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3303. Min Act: -0.1640

Data Index: 8137924 (Open Web Text)

Max Activating Token Index: 707

Text #8

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3304. Min Act: -0.1640

Data Index: 1689443 (Open Web Text)

Max Activating Token Index: 748

Text #9

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3336. Min Act: -0.1640

Data Index: 6754110 (Open Web Text)

Max Activating Token Index: 981

Text #10

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3260. Min Act: -0.1640

Data Index: 4445889 (Open Web Text)

Max Activating Token Index: 1002

Text #11

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3232. Min Act: -0.1640

Data Index: 7537547 (Open Web Text)

Max Activating Token Index: 955

Text #12

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3151. Min Act: -0.1640

Data Index: 3416377 (Open Web Text)

Max Activating Token Index: 947

Text #13

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3135. Min Act: -0.1640

Data Index: 2644815 (Open Web Text)

Max Activating Token Index: 940

Text #14

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3191. Min Act: -0.1640

Data Index: 1743895 (Open Web Text)

Max Activating Token Index: 928

Text #15

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3225. Min Act: -0.1640

Data Index: 5667563 (Open Web Text)

Max Activating Token Index: 992

Text #16

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3163. Min Act: -0.1640

Data Index: 1580417 (Open Web Text)

Max Activating Token Index: 921

Text #17

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3140. Min Act: -0.1640

Data Index: 5047576 (Open Web Text)

Max Activating Token Index: 837

Text #18

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3119. Min Act: -0.1640

Data Index: 4158383 (Open Web Text)

Max Activating Token Index: 345

Text #19

Max Range: 1.3695. Min Range: -1.3695

Max Act: 1.3074. Min Act: -0.1640

Data Index: 231207 (Open Web Text)

Max Activating Token Index: 286