Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1407 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.1461. Min Act: -0.1700

Data Index: 4836860 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.1423. Min Act: -0.1700

Data Index: 4498414 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.1391. Min Act: -0.1700

Data Index: 2687001 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.1149. Min Act: -0.1700

Data Index: 3636422 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.1022. Min Act: -0.1700

Data Index: 7169679 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0887. Min Act: -0.1700

Data Index: 6862803 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0928. Min Act: -0.1700

Data Index: 5507640 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0877. Min Act: -0.1700

Data Index: 3796601 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0871. Min Act: -0.1700

Data Index: 6437319 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.1034. Min Act: -0.1700

Data Index: 4320132 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0841. Min Act: -0.1700

Data Index: 1355120 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0724. Min Act: -0.1700

Data Index: 8354213 (Open Web Text)

Max Activating Token Index: 2

Text #12

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0841. Min Act: -0.1700

Data Index: 6183588 (Open Web Text)

Max Activating Token Index: 2

Text #13

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0761. Min Act: -0.1700

Data Index: 7304532 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0762. Min Act: -0.1700

Data Index: 6435056 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0887. Min Act: -0.1700

Data Index: 686005 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0768. Min Act: -0.1700

Data Index: 6891189 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0613. Min Act: -0.1700

Data Index: 1157440 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0715. Min Act: -0.1700

Data Index: 1496668 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 3.1461. Min Range: -3.1461

Max Act: 3.0595. Min Act: -0.1700

Data Index: 1923432 (Open Web Text)

Max Activating Token Index: 11