Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 726 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.5149. Min Act: -0.1700

Data Index: 2868172 (Open Web Text)

Max Activating Token Index: 981

Text #1

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.4141. Min Act: -0.1700

Data Index: 5745861 (Open Web Text)

Max Activating Token Index: 331

Text #2

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.3726. Min Act: -0.1700

Data Index: 749750 (Open Web Text)

Max Activating Token Index: 775

Text #3

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.4159. Min Act: -0.1700

Data Index: 3755704 (Open Web Text)

Max Activating Token Index: 163

Text #4

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.4013. Min Act: -0.1700

Data Index: 7677322 (Open Web Text)

Max Activating Token Index: 893

Text #5

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.3667. Min Act: -0.1700

Data Index: 2623772 (Open Web Text)

Max Activating Token Index: 614

Text #6

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.3167. Min Act: -0.1700

Data Index: 1170987 (Open Web Text)

Max Activating Token Index: 828

Text #7

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.3200. Min Act: -0.1700

Data Index: 2141660 (Open Web Text)

Max Activating Token Index: 198

Text #8

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2973. Min Act: -0.1700

Data Index: 1608708 (Open Web Text)

Max Activating Token Index: 689

Text #9

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.3106. Min Act: -0.1700

Data Index: 7602583 (Open Web Text)

Max Activating Token Index: 217

Text #10

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2739. Min Act: -0.1700

Data Index: 1825124 (Open Web Text)

Max Activating Token Index: 633

Text #11

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2654. Min Act: -0.1700

Data Index: 1073504 (Open Web Text)

Max Activating Token Index: 274

Text #12

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2491. Min Act: -0.1700

Data Index: 5648109 (Open Web Text)

Max Activating Token Index: 823

Text #13

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2527. Min Act: -0.1700

Data Index: 7038546 (Open Web Text)

Max Activating Token Index: 352

Text #14

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2836. Min Act: -0.1700

Data Index: 4069000 (Open Web Text)

Max Activating Token Index: 954

Text #15

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2373. Min Act: -0.1700

Data Index: 3711690 (Open Web Text)

Max Activating Token Index: 288

Text #16

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2188. Min Act: -0.1700

Data Index: 1981847 (Open Web Text)

Max Activating Token Index: 256

Text #17

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2423. Min Act: -0.1700

Data Index: 4477387 (Open Web Text)

Max Activating Token Index: 912

Text #18

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2260. Min Act: -0.1700

Data Index: 5645888 (Open Web Text)

Max Activating Token Index: 318

Text #19

Max Range: 4.5149. Min Range: -4.5149

Max Act: 4.2025. Min Act: -0.1700

Data Index: 4337074 (Open Web Text)

Max Activating Token Index: 965