Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 787 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.7946. Min Act: -0.1700

Data Index: 28996 (Open Web Text)

Max Activating Token Index: 695

Text #1

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.7348. Min Act: -0.1700

Data Index: 2087959 (Open Web Text)

Max Activating Token Index: 517

Text #2

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4982. Min Act: -0.1700

Data Index: 1915209 (Open Web Text)

Max Activating Token Index: 990

Text #3

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.5081. Min Act: -0.1700

Data Index: 5698166 (Open Web Text)

Max Activating Token Index: 814

Text #4

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.5337. Min Act: -0.1700

Data Index: 5076902 (Open Web Text)

Max Activating Token Index: 920

Text #5

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4952. Min Act: -0.1700

Data Index: 4894018 (Open Web Text)

Max Activating Token Index: 754

Text #6

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4997. Min Act: -0.1700

Data Index: 6131767 (Open Web Text)

Max Activating Token Index: 747

Text #7

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4880. Min Act: -0.1700

Data Index: 716712 (Open Web Text)

Max Activating Token Index: 28

Text #8

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4704. Min Act: -0.1700

Data Index: 7750945 (Open Web Text)

Max Activating Token Index: 408

Text #9

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4482. Min Act: -0.1700

Data Index: 6069733 (Open Web Text)

Max Activating Token Index: 123

Text #10

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4366. Min Act: -0.1700

Data Index: 6027866 (Open Web Text)

Max Activating Token Index: 257

Text #11

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4840. Min Act: -0.1700

Data Index: 4199241 (Open Web Text)

Max Activating Token Index: 508

Text #12

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4227. Min Act: -0.1700

Data Index: 2839828 (Open Web Text)

Max Activating Token Index: 433

Text #13

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4577. Min Act: -0.1700

Data Index: 3880400 (Open Web Text)

Max Activating Token Index: 941

Text #14

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4573. Min Act: -0.1700

Data Index: 309602 (Open Web Text)

Max Activating Token Index: 343

Text #15

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4118. Min Act: -0.1700

Data Index: 2781670 (Open Web Text)

Max Activating Token Index: 364

Text #16

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.3633. Min Act: -0.1700

Data Index: 8249796 (Open Web Text)

Max Activating Token Index: 1015

Text #17

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.3768. Min Act: -0.1700

Data Index: 5255616 (Open Web Text)

Max Activating Token Index: 432

Text #18

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.3987. Min Act: -0.1700

Data Index: 2030975 (Open Web Text)

Max Activating Token Index: 354

Text #19

Max Range: 6.7946. Min Range: -6.7946

Max Act: 6.4176. Min Act: -0.1700

Data Index: 2206488 (Open Web Text)

Max Activating Token Index: 946