Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 616 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.5158. Min Act: -0.1700

Data Index: 8420729 (Open Web Text)

Max Activating Token Index: 415

Text #1

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.5182. Min Act: -0.1700

Data Index: 5213256 (Open Web Text)

Max Activating Token Index: 849

Text #2

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.4676. Min Act: -0.1700

Data Index: 7787346 (Open Web Text)

Max Activating Token Index: 453

Text #3

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.4489. Min Act: -0.1700

Data Index: 5363732 (Open Web Text)

Max Activating Token Index: 810

Text #4

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3943. Min Act: -0.1700

Data Index: 6714004 (Open Web Text)

Max Activating Token Index: 932

Text #5

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.4370. Min Act: -0.1700

Data Index: 1694466 (Open Web Text)

Max Activating Token Index: 432

Text #6

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3882. Min Act: -0.1700

Data Index: 3868480 (Open Web Text)

Max Activating Token Index: 622

Text #7

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3942. Min Act: -0.1700

Data Index: 6775984 (Open Web Text)

Max Activating Token Index: 629

Text #8

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3887. Min Act: -0.1700

Data Index: 1426240 (Open Web Text)

Max Activating Token Index: 651

Text #9

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3817. Min Act: -0.1700

Data Index: 1837991 (Open Web Text)

Max Activating Token Index: 249

Text #10

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3420. Min Act: -0.1700

Data Index: 5222033 (Open Web Text)

Max Activating Token Index: 545

Text #11

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3690. Min Act: -0.1700

Data Index: 1074397 (Open Web Text)

Max Activating Token Index: 840

Text #12

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3603. Min Act: -0.1700

Data Index: 1666436 (Open Web Text)

Max Activating Token Index: 284

Text #13

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3372. Min Act: -0.1700

Data Index: 460641 (Open Web Text)

Max Activating Token Index: 26

Text #14

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3311. Min Act: -0.1700

Data Index: 4769094 (Open Web Text)

Max Activating Token Index: 565

Text #15

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3442. Min Act: -0.1700

Data Index: 3787881 (Open Web Text)

Max Activating Token Index: 440

Text #16

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3347. Min Act: -0.1700

Data Index: 694884 (Open Web Text)

Max Activating Token Index: 743

Text #17

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3272. Min Act: -0.1700

Data Index: 7154102 (Open Web Text)

Max Activating Token Index: 57

Text #18

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3248. Min Act: -0.1700

Data Index: 2724723 (Open Web Text)

Max Activating Token Index: 771

Text #19

Max Range: 3.5182. Min Range: -3.5182

Max Act: 3.3282. Min Act: -0.1700

Data Index: 3669057 (Open Web Text)

Max Activating Token Index: 695