Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2440 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2828. Min Act: -0.1011

Data Index: 8625707 (Open Web Text)

Max Activating Token Index: 648

Text #1

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2779. Min Act: -0.0603

Data Index: 807355 (Open Web Text)

Max Activating Token Index: 640

Text #2

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2688. Min Act: -0.1317

Data Index: 4488772 (Open Web Text)

Max Activating Token Index: 694

Text #3

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2616. Min Act: -0.0308

Data Index: 3710162 (Open Web Text)

Max Activating Token Index: 1018

Text #4

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2690. Min Act: -0.0586

Data Index: 3471031 (Open Web Text)

Max Activating Token Index: 670

Text #5

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2482. Min Act: -0.0753

Data Index: 6595413 (Open Web Text)

Max Activating Token Index: 613

Text #6

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2539. Min Act: -0.0166

Data Index: 4759293 (Open Web Text)

Max Activating Token Index: 23

Text #7

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2491. Min Act: -0.0824

Data Index: 658595 (Open Web Text)

Max Activating Token Index: 809

Text #8

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2362. Min Act: -0.1157

Data Index: 5649845 (Open Web Text)

Max Activating Token Index: 231

Text #9

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2291. Min Act: -0.0810

Data Index: 4759277 (Open Web Text)

Max Activating Token Index: 227

Text #10

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2289. Min Act: -0.0813

Data Index: 2142372 (Open Web Text)

Max Activating Token Index: 1007

Text #11

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2267. Min Act: -0.0610

Data Index: 4137705 (Open Web Text)

Max Activating Token Index: 849

Text #12

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2294. Min Act: -0.0166

Data Index: 773003 (Open Web Text)

Max Activating Token Index: 715

Text #13

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2239. Min Act: -0.0408

Data Index: 1446715 (Open Web Text)

Max Activating Token Index: 628

Text #14

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2165. Min Act: -0.0485

Data Index: 4718342 (Open Web Text)

Max Activating Token Index: 838

Text #15

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2169. Min Act: -0.1352

Data Index: 3751525 (Open Web Text)

Max Activating Token Index: 555

Text #16

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2217. Min Act: -0.0681

Data Index: 1680630 (Open Web Text)

Max Activating Token Index: 681

Text #17

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2197. Min Act: -0.0265

Data Index: 7047386 (Open Web Text)

Max Activating Token Index: 23

Text #18

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2235. Min Act: -0.0944

Data Index: 1743449 (Open Web Text)

Max Activating Token Index: 705

Text #19

Max Range: 1.2828. Min Range: -1.2828

Max Act: 1.2143. Min Act: -0.0507

Data Index: 3037669 (Open Web Text)

Max Activating Token Index: 723