Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1924 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4857. Min Act: -0.1700

Data Index: 8360710 (Open Web Text)

Max Activating Token Index: 900

Text #1

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4808. Min Act: -0.1700

Data Index: 6358616 (Open Web Text)

Max Activating Token Index: 146

Text #2

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4706. Min Act: -0.1700

Data Index: 5748765 (Open Web Text)

Max Activating Token Index: 539

Text #3

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4615. Min Act: -0.1700

Data Index: 7319585 (Open Web Text)

Max Activating Token Index: 676

Text #4

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4637. Min Act: -0.1700

Data Index: 1312154 (Open Web Text)

Max Activating Token Index: 516

Text #5

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4345. Min Act: -0.1700

Data Index: 3960340 (Open Web Text)

Max Activating Token Index: 188

Text #6

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4401. Min Act: -0.1700

Data Index: 1344634 (Open Web Text)

Max Activating Token Index: 523

Text #7

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4165. Min Act: -0.1700

Data Index: 2493574 (Open Web Text)

Max Activating Token Index: 238

Text #8

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4227. Min Act: -0.1700

Data Index: 1417096 (Open Web Text)

Max Activating Token Index: 107

Text #9

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4191. Min Act: -0.1700

Data Index: 6935442 (Open Web Text)

Max Activating Token Index: 593

Text #10

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4173. Min Act: -0.1700

Data Index: 5581094 (Open Web Text)

Max Activating Token Index: 347

Text #11

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4148. Min Act: -0.1700

Data Index: 6484643 (Open Web Text)

Max Activating Token Index: 291

Text #12

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4093. Min Act: -0.1700

Data Index: 2612380 (Open Web Text)

Max Activating Token Index: 515

Text #13

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4130. Min Act: -0.1700

Data Index: 4465052 (Open Web Text)

Max Activating Token Index: 946

Text #14

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4015. Min Act: -0.1700

Data Index: 6638061 (Open Web Text)

Max Activating Token Index: 63

Text #15

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4130. Min Act: -0.1700

Data Index: 2162807 (Open Web Text)

Max Activating Token Index: 290

Text #16

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4163. Min Act: -0.1700

Data Index: 1195122 (Open Web Text)

Max Activating Token Index: 304

Text #17

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4009. Min Act: -0.1700

Data Index: 4222418 (Open Web Text)

Max Activating Token Index: 864

Text #18

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4050. Min Act: -0.1700

Data Index: 3750238 (Open Web Text)

Max Activating Token Index: 214

Text #19

Max Range: 2.4857. Min Range: -2.4857

Max Act: 2.4087. Min Act: -0.1700

Data Index: 576209 (Open Web Text)

Max Activating Token Index: 146