Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 436 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.4695. Min Act: -0.1700

Data Index: 7953788 (Open Web Text)

Max Activating Token Index: 1018

Text #1

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.4092. Min Act: -0.1700

Data Index: 5926741 (Open Web Text)

Max Activating Token Index: 1021

Text #2

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.3775. Min Act: -0.1700

Data Index: 6792256 (Open Web Text)

Max Activating Token Index: 614

Text #3

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.3791. Min Act: -0.1700

Data Index: 6982013 (Open Web Text)

Max Activating Token Index: 742

Text #4

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.3498. Min Act: -0.1700

Data Index: 4690586 (Open Web Text)

Max Activating Token Index: 555

Text #5

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.3468. Min Act: -0.1700

Data Index: 4303637 (Open Web Text)

Max Activating Token Index: 168

Text #6

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.3186. Min Act: -0.1700

Data Index: 8066097 (Open Web Text)

Max Activating Token Index: 960

Text #7

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.3188. Min Act: -0.1700

Data Index: 6639204 (Open Web Text)

Max Activating Token Index: 428

Text #8

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2989. Min Act: -0.1700

Data Index: 1073716 (Open Web Text)

Max Activating Token Index: 529

Text #9

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2998. Min Act: -0.1700

Data Index: 3420807 (Open Web Text)

Max Activating Token Index: 797

Text #10

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2862. Min Act: -0.1700

Data Index: 7415423 (Open Web Text)

Max Activating Token Index: 101

Text #11

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2898. Min Act: -0.1700

Data Index: 7950768 (Open Web Text)

Max Activating Token Index: 598

Text #12

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2721. Min Act: -0.1700

Data Index: 6290223 (Open Web Text)

Max Activating Token Index: 692

Text #13

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2753. Min Act: -0.1700

Data Index: 8496119 (Open Web Text)

Max Activating Token Index: 915

Text #14

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2720. Min Act: -0.1700

Data Index: 1284431 (Open Web Text)

Max Activating Token Index: 697

Text #15

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2316. Min Act: -0.1700

Data Index: 8012263 (Open Web Text)

Max Activating Token Index: 985

Text #16

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2556. Min Act: -0.1700

Data Index: 3871329 (Open Web Text)

Max Activating Token Index: 59

Text #17

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2334. Min Act: -0.1700

Data Index: 1064999 (Open Web Text)

Max Activating Token Index: 308

Text #18

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2383. Min Act: -0.1700

Data Index: 7585130 (Open Web Text)

Max Activating Token Index: 344

Text #19

Max Range: 3.4695. Min Range: -3.4695

Max Act: 3.2260. Min Act: -0.1700

Data Index: 7343742 (Open Web Text)

Max Activating Token Index: 655