Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1268 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.6754. Min Act: -0.1700

Data Index: 742995 (Open Web Text)

Max Activating Token Index: 412

Text #1

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.6112. Min Act: -0.1700

Data Index: 3509623 (Open Web Text)

Max Activating Token Index: 300

Text #2

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.6259. Min Act: -0.1700

Data Index: 2394824 (Open Web Text)

Max Activating Token Index: 936

Text #3

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.5622. Min Act: -0.1700

Data Index: 7103688 (Open Web Text)

Max Activating Token Index: 352

Text #4

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.5438. Min Act: -0.1700

Data Index: 344461 (Open Web Text)

Max Activating Token Index: 36

Text #5

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4664. Min Act: -0.1700

Data Index: 5015854 (Open Web Text)

Max Activating Token Index: 595

Text #6

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4905. Min Act: -0.1700

Data Index: 5766600 (Open Web Text)

Max Activating Token Index: 551

Text #7

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4698. Min Act: -0.1700

Data Index: 7941717 (Open Web Text)

Max Activating Token Index: 327

Text #8

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4629. Min Act: -0.1700

Data Index: 2125188 (Open Web Text)

Max Activating Token Index: 397

Text #9

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4692. Min Act: -0.1700

Data Index: 6924337 (Open Web Text)

Max Activating Token Index: 47

Text #10

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4568. Min Act: -0.1700

Data Index: 4275980 (Open Web Text)

Max Activating Token Index: 369

Text #11

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4317. Min Act: -0.1700

Data Index: 6422183 (Open Web Text)

Max Activating Token Index: 302

Text #12

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4432. Min Act: -0.1700

Data Index: 8004835 (Open Web Text)

Max Activating Token Index: 633

Text #13

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4334. Min Act: -0.1700

Data Index: 4989887 (Open Web Text)

Max Activating Token Index: 175

Text #14

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4412. Min Act: -0.1700

Data Index: 8746555 (Open Web Text)

Max Activating Token Index: 489

Text #15

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.3972. Min Act: -0.1700

Data Index: 1159448 (Open Web Text)

Max Activating Token Index: 575

Text #16

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4202. Min Act: -0.1700

Data Index: 6751926 (Open Web Text)

Max Activating Token Index: 838

Text #17

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4027. Min Act: -0.1700

Data Index: 6575079 (Open Web Text)

Max Activating Token Index: 658

Text #18

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4111. Min Act: -0.1700

Data Index: 5450508 (Open Web Text)

Max Activating Token Index: 575

Text #19

Max Range: 4.6754. Min Range: -4.6754

Max Act: 4.4004. Min Act: -0.1700

Data Index: 3650960 (Open Web Text)

Max Activating Token Index: 278