Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 925 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0821. Min Range: -4.0821

Max Act: 4.0821. Min Act: -0.1700

Data Index: 6881004 (Open Web Text)

Max Activating Token Index: 968

Text #1

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.8428. Min Act: -0.1700

Data Index: 1345169 (Open Web Text)

Max Activating Token Index: 44

Text #2

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7851. Min Act: -0.1700

Data Index: 4212603 (Open Web Text)

Max Activating Token Index: 603

Text #3

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7942. Min Act: -0.1700

Data Index: 3185328 (Open Web Text)

Max Activating Token Index: 732

Text #4

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7662. Min Act: -0.1700

Data Index: 892682 (Open Web Text)

Max Activating Token Index: 957

Text #5

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7534. Min Act: -0.1700

Data Index: 3379094 (Open Web Text)

Max Activating Token Index: 500

Text #6

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7301. Min Act: -0.1700

Data Index: 3329823 (Open Web Text)

Max Activating Token Index: 294

Text #7

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7009. Min Act: -0.1700

Data Index: 2787702 (Open Web Text)

Max Activating Token Index: 128

Text #8

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6811. Min Act: -0.1700

Data Index: 6687537 (Open Web Text)

Max Activating Token Index: 319

Text #9

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6961. Min Act: -0.1700

Data Index: 2368243 (Open Web Text)

Max Activating Token Index: 380

Text #10

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.7131. Min Act: -0.1700

Data Index: 455482 (Open Web Text)

Max Activating Token Index: 943

Text #11

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6628. Min Act: -0.1700

Data Index: 5659547 (Open Web Text)

Max Activating Token Index: 727

Text #12

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6427. Min Act: -0.1700

Data Index: 7313378 (Open Web Text)

Max Activating Token Index: 113

Text #13

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6397. Min Act: -0.1700

Data Index: 3780056 (Open Web Text)

Max Activating Token Index: 471

Text #14

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6477. Min Act: -0.1700

Data Index: 6687540 (Open Web Text)

Max Activating Token Index: 461

Text #15

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6814. Min Act: -0.1700

Data Index: 6753737 (Open Web Text)

Max Activating Token Index: 486

Text #16

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6664. Min Act: -0.1700

Data Index: 2148706 (Open Web Text)

Max Activating Token Index: 309

Text #17

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6734. Min Act: -0.1700

Data Index: 2097217 (Open Web Text)

Max Activating Token Index: 762

Text #18

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6772. Min Act: -0.1700

Data Index: 8528000 (Open Web Text)

Max Activating Token Index: 425

Text #19

Max Range: 4.0821. Min Range: -4.0821

Max Act: 3.6187. Min Act: -0.1700

Data Index: 4100970 (Open Web Text)

Max Activating Token Index: 385