Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1924 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.6078. Min Act: -0.1700

Data Index: 7410040 (Open Web Text)

Max Activating Token Index: 18

Text #1

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5989. Min Act: -0.1700

Data Index: 4658953 (Open Web Text)

Max Activating Token Index: 14

Text #2

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5857. Min Act: -0.1700

Data Index: 3509716 (Open Web Text)

Max Activating Token Index: 29

Text #3

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5816. Min Act: -0.1700

Data Index: 2615216 (Open Web Text)

Max Activating Token Index: 26

Text #4

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5710. Min Act: -0.1700

Data Index: 1248238 (Open Web Text)

Max Activating Token Index: 23

Text #5

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5491. Min Act: -0.1700

Data Index: 1331998 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5458. Min Act: -0.1700

Data Index: 1617641 (Open Web Text)

Max Activating Token Index: 403

Text #7

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5510. Min Act: -0.1700

Data Index: 5053447 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5512. Min Act: -0.1700

Data Index: 5456472 (Open Web Text)

Max Activating Token Index: 612

Text #9

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5494. Min Act: -0.1700

Data Index: 1979489 (Open Web Text)

Max Activating Token Index: 35

Text #10

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5471. Min Act: -0.1700

Data Index: 6617454 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5463. Min Act: -0.1700

Data Index: 2182870 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5570. Min Act: -0.1700

Data Index: 4320757 (Open Web Text)

Max Activating Token Index: 8

Text #13

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5721. Min Act: -0.1700

Data Index: 8019396 (Open Web Text)

Max Activating Token Index: 254

Text #14

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5492. Min Act: -0.1700

Data Index: 899447 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5453. Min Act: -0.1700

Data Index: 8446210 (Open Web Text)

Max Activating Token Index: 451

Text #16

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5515. Min Act: -0.1700

Data Index: 8585265 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5631. Min Act: -0.1700

Data Index: 2825783 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5207. Min Act: -0.1700

Data Index: 2214963 (Open Web Text)

Max Activating Token Index: 366

Text #19

Max Range: 3.6078. Min Range: -3.6078

Max Act: 3.5093. Min Act: -0.1700

Data Index: 2579422 (Open Web Text)

Max Activating Token Index: 652