Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2454 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.7085. Min Act: -0.1700

Data Index: 8414511 (Open Web Text)

Max Activating Token Index: 218

Text #1

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.6443. Min Act: -0.1700

Data Index: 5219868 (Open Web Text)

Max Activating Token Index: 750

Text #2

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.6562. Min Act: -0.1700

Data Index: 3096366 (Open Web Text)

Max Activating Token Index: 121

Text #3

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.5727. Min Act: -0.1700

Data Index: 6744701 (Open Web Text)

Max Activating Token Index: 642

Text #4

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.5219. Min Act: -0.1700

Data Index: 4136425 (Open Web Text)

Max Activating Token Index: 96

Text #5

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.5171. Min Act: -0.1700

Data Index: 3312091 (Open Web Text)

Max Activating Token Index: 724

Text #6

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.5517. Min Act: -0.1700

Data Index: 7224169 (Open Web Text)

Max Activating Token Index: 863

Text #7

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.5197. Min Act: -0.1700

Data Index: 6646757 (Open Web Text)

Max Activating Token Index: 221

Text #8

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4676. Min Act: -0.1700

Data Index: 7334956 (Open Web Text)

Max Activating Token Index: 344

Text #9

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4729. Min Act: -0.1700

Data Index: 5036963 (Open Web Text)

Max Activating Token Index: 564

Text #10

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4440. Min Act: -0.1700

Data Index: 7805540 (Open Web Text)

Max Activating Token Index: 603

Text #11

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4536. Min Act: -0.1700

Data Index: 5588543 (Open Web Text)

Max Activating Token Index: 297

Text #12

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4418. Min Act: -0.1700

Data Index: 2809560 (Open Web Text)

Max Activating Token Index: 618

Text #13

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4506. Min Act: -0.1700

Data Index: 2058695 (Open Web Text)

Max Activating Token Index: 844

Text #14

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.3967. Min Act: -0.1700

Data Index: 3968471 (Open Web Text)

Max Activating Token Index: 270

Text #15

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4019. Min Act: -0.1700

Data Index: 7600579 (Open Web Text)

Max Activating Token Index: 857

Text #16

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.3878. Min Act: -0.1700

Data Index: 8404292 (Open Web Text)

Max Activating Token Index: 68

Text #17

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4145. Min Act: -0.1700

Data Index: 8358342 (Open Web Text)

Max Activating Token Index: 1000

Text #18

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4086. Min Act: -0.1700

Data Index: 2182471 (Open Web Text)

Max Activating Token Index: 699

Text #19

Max Range: 4.7085. Min Range: -4.7085

Max Act: 4.4117. Min Act: -0.1700

Data Index: 6799226 (Open Web Text)

Max Activating Token Index: 297