Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2405 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.8188. Min Act: -0.1700

Data Index: 3614201 (Open Web Text)

Max Activating Token Index: 703

Text #1

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.8259. Min Act: -0.1700

Data Index: 4064993 (Open Web Text)

Max Activating Token Index: 517

Text #2

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.8166. Min Act: -0.1700

Data Index: 4492200 (Open Web Text)

Max Activating Token Index: 417

Text #3

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.8140. Min Act: -0.1700

Data Index: 709257 (Open Web Text)

Max Activating Token Index: 631

Text #4

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7519. Min Act: -0.1700

Data Index: 303376 (Open Web Text)

Max Activating Token Index: 554

Text #5

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7620. Min Act: -0.1700

Data Index: 7471857 (Open Web Text)

Max Activating Token Index: 414

Text #6

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7429. Min Act: -0.1700

Data Index: 4641188 (Open Web Text)

Max Activating Token Index: 602

Text #7

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7471. Min Act: -0.1700

Data Index: 902835 (Open Web Text)

Max Activating Token Index: 447

Text #8

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7186. Min Act: -0.1700

Data Index: 2606079 (Open Web Text)

Max Activating Token Index: 300

Text #9

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7354. Min Act: -0.1700

Data Index: 7895816 (Open Web Text)

Max Activating Token Index: 458

Text #10

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7384. Min Act: -0.1700

Data Index: 3450164 (Open Web Text)

Max Activating Token Index: 694

Text #11

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7141. Min Act: -0.1700

Data Index: 5501829 (Open Web Text)

Max Activating Token Index: 390

Text #12

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.6854. Min Act: -0.1700

Data Index: 448029 (Open Web Text)

Max Activating Token Index: 670

Text #13

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.6913. Min Act: -0.1700

Data Index: 2043875 (Open Web Text)

Max Activating Token Index: 443

Text #14

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7062. Min Act: -0.1700

Data Index: 6429894 (Open Web Text)

Max Activating Token Index: 812

Text #15

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7200. Min Act: -0.1700

Data Index: 4088829 (Open Web Text)

Max Activating Token Index: 975

Text #16

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.6841. Min Act: -0.1700

Data Index: 6057859 (Open Web Text)

Max Activating Token Index: 102

Text #17

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.6944. Min Act: -0.1700

Data Index: 6583684 (Open Web Text)

Max Activating Token Index: 140

Text #18

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.6925. Min Act: -0.1700

Data Index: 595508 (Open Web Text)

Max Activating Token Index: 727

Text #19

Max Range: 3.8259. Min Range: -3.8259

Max Act: 3.7155. Min Act: -0.1700

Data Index: 5677742 (Open Web Text)

Max Activating Token Index: 455