Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1843 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.9694. Min Act: -0.1700

Data Index: 6453412 (Open Web Text)

Max Activating Token Index: 185

Text #1

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.8873. Min Act: -0.1700

Data Index: 2349996 (Open Web Text)

Max Activating Token Index: 440

Text #2

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.9255. Min Act: -0.1700

Data Index: 2880551 (Open Web Text)

Max Activating Token Index: 883

Text #3

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.9464. Min Act: -0.1700

Data Index: 7399295 (Open Web Text)

Max Activating Token Index: 465

Text #4

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.9306. Min Act: -0.1700

Data Index: 1983008 (Open Web Text)

Max Activating Token Index: 706

Text #5

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.8221. Min Act: -0.1700

Data Index: 6453411 (Open Web Text)

Max Activating Token Index: 603

Text #6

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.8347. Min Act: -0.1700

Data Index: 2713594 (Open Web Text)

Max Activating Token Index: 702

Text #7

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.8870. Min Act: -0.1700

Data Index: 5609370 (Open Web Text)

Max Activating Token Index: 450

Text #8

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.8173. Min Act: -0.1700

Data Index: 4361635 (Open Web Text)

Max Activating Token Index: 250

Text #9

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.8443. Min Act: -0.1700

Data Index: 740861 (Open Web Text)

Max Activating Token Index: 102

Text #10

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7664. Min Act: -0.1700

Data Index: 6346317 (Open Web Text)

Max Activating Token Index: 292

Text #11

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7760. Min Act: -0.1700

Data Index: 3694621 (Open Web Text)

Max Activating Token Index: 56

Text #12

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7622. Min Act: -0.1700

Data Index: 6683581 (Open Web Text)

Max Activating Token Index: 754

Text #13

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7622. Min Act: -0.1700

Data Index: 6577559 (Open Web Text)

Max Activating Token Index: 42

Text #14

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7315. Min Act: -0.1700

Data Index: 250751 (Open Web Text)

Max Activating Token Index: 453

Text #15

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7696. Min Act: -0.1700

Data Index: 6860431 (Open Web Text)

Max Activating Token Index: 947

Text #16

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7380. Min Act: -0.1700

Data Index: 5178385 (Open Web Text)

Max Activating Token Index: 951

Text #17

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7477. Min Act: -0.1700

Data Index: 3139 (Open Web Text)

Max Activating Token Index: 480

Text #18

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.7330. Min Act: -0.1700

Data Index: 3359325 (Open Web Text)

Max Activating Token Index: 639

Text #19

Max Range: 4.9694. Min Range: -4.9694

Max Act: 4.6994. Min Act: -0.1700

Data Index: 7116255 (Open Web Text)

Max Activating Token Index: 443