Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 835 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.8423. Min Act: -0.1700

Data Index: 6253984 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.8423. Min Act: -0.1700

Data Index: 287499 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7958. Min Act: -0.1700

Data Index: 8511155 (Open Web Text)

Max Activating Token Index: 10

Text #3

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7793. Min Act: -0.1700

Data Index: 4617159 (Open Web Text)

Max Activating Token Index: 12

Text #4

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7801. Min Act: -0.1700

Data Index: 3970492 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7659. Min Act: -0.1700

Data Index: 5657506 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7507. Min Act: -0.1700

Data Index: 6456189 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7364. Min Act: -0.1700

Data Index: 3713736 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7387. Min Act: -0.1700

Data Index: 6173372 (Open Web Text)

Max Activating Token Index: 8

Text #9

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7321. Min Act: -0.1700

Data Index: 5472415 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7364. Min Act: -0.1700

Data Index: 7944408 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7331. Min Act: -0.1700

Data Index: 7287109 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7108. Min Act: -0.1700

Data Index: 8707572 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7212. Min Act: -0.1700

Data Index: 7110373 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7100. Min Act: -0.1700

Data Index: 7086589 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.6959. Min Act: -0.1700

Data Index: 1698173 (Open Web Text)

Max Activating Token Index: 33

Text #16

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7042. Min Act: -0.1700

Data Index: 4089721 (Open Web Text)

Max Activating Token Index: 10

Text #17

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7083. Min Act: -0.1700

Data Index: 2313182 (Open Web Text)

Max Activating Token Index: 573

Text #18

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.7020. Min Act: -0.1700

Data Index: 1550873 (Open Web Text)

Max Activating Token Index: 9

Text #19

Max Range: 1.8423. Min Range: -1.8423

Max Act: 1.6973. Min Act: -0.1700

Data Index: 5922857 (Open Web Text)

Max Activating Token Index: 8