Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 227 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.8258. Min Act: -0.1700

Data Index: 6312738 (Open Web Text)

Max Activating Token Index: 961

Text #1

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.6999. Min Act: -0.1700

Data Index: 5812452 (Open Web Text)

Max Activating Token Index: 408

Text #2

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.5677. Min Act: -0.1700

Data Index: 7215138 (Open Web Text)

Max Activating Token Index: 678

Text #3

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.5546. Min Act: -0.1700

Data Index: 2131447 (Open Web Text)

Max Activating Token Index: 281

Text #4

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.5435. Min Act: -0.1700

Data Index: 3088486 (Open Web Text)

Max Activating Token Index: 616

Text #5

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.5306. Min Act: -0.1700

Data Index: 4736136 (Open Web Text)

Max Activating Token Index: 828

Text #6

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.4961. Min Act: -0.1700

Data Index: 1981677 (Open Web Text)

Max Activating Token Index: 825

Text #7

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.4435. Min Act: -0.1700

Data Index: 8056964 (Open Web Text)

Max Activating Token Index: 764

Text #8

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.4025. Min Act: -0.1700

Data Index: 508866 (Open Web Text)

Max Activating Token Index: 370

Text #9

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3763. Min Act: -0.1700

Data Index: 4400971 (Open Web Text)

Max Activating Token Index: 845

Text #10

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.4141. Min Act: -0.1700

Data Index: 4646020 (Open Web Text)

Max Activating Token Index: 447

Text #11

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3924. Min Act: -0.1700

Data Index: 1339239 (Open Web Text)

Max Activating Token Index: 518

Text #12

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3399. Min Act: -0.1700

Data Index: 8309740 (Open Web Text)

Max Activating Token Index: 970

Text #13

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3844. Min Act: -0.1700

Data Index: 5098312 (Open Web Text)

Max Activating Token Index: 787

Text #14

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3446. Min Act: -0.1700

Data Index: 4214073 (Open Web Text)

Max Activating Token Index: 731

Text #15

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.2952. Min Act: -0.1700

Data Index: 7845843 (Open Web Text)

Max Activating Token Index: 396

Text #16

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3617. Min Act: -0.1700

Data Index: 4391398 (Open Web Text)

Max Activating Token Index: 988

Text #17

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3168. Min Act: -0.1700

Data Index: 2727745 (Open Web Text)

Max Activating Token Index: 621

Text #18

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3323. Min Act: -0.1700

Data Index: 298696 (Open Web Text)

Max Activating Token Index: 920

Text #19

Max Range: 6.8258. Min Range: -6.8258

Max Act: 6.3270. Min Act: -0.1700

Data Index: 4452892 (Open Web Text)

Max Activating Token Index: 981