Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1117 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.8666. Min Act: -0.1700

Data Index: 7476639 (Open Web Text)

Max Activating Token Index: 623

Text #1

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.8182. Min Act: -0.1700

Data Index: 439478 (Open Web Text)

Max Activating Token Index: 94

Text #2

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.7730. Min Act: -0.1700

Data Index: 1340958 (Open Web Text)

Max Activating Token Index: 103

Text #3

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.7465. Min Act: -0.1700

Data Index: 5915115 (Open Web Text)

Max Activating Token Index: 557

Text #4

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.7437. Min Act: -0.1700

Data Index: 6442794 (Open Web Text)

Max Activating Token Index: 444

Text #5

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.7073. Min Act: -0.1700

Data Index: 2900672 (Open Web Text)

Max Activating Token Index: 621

Text #6

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6677. Min Act: -0.1700

Data Index: 3198701 (Open Web Text)

Max Activating Token Index: 556

Text #7

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6453. Min Act: -0.1700

Data Index: 1891877 (Open Web Text)

Max Activating Token Index: 864

Text #8

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6691. Min Act: -0.1700

Data Index: 4607806 (Open Web Text)

Max Activating Token Index: 224

Text #9

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6447. Min Act: -0.1700

Data Index: 3089674 (Open Web Text)

Max Activating Token Index: 48

Text #10

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6493. Min Act: -0.1700

Data Index: 7467248 (Open Web Text)

Max Activating Token Index: 578

Text #11

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6461. Min Act: -0.1700

Data Index: 123787 (Open Web Text)

Max Activating Token Index: 415

Text #12

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6579. Min Act: -0.1700

Data Index: 2066152 (Open Web Text)

Max Activating Token Index: 493

Text #13

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6079. Min Act: -0.1700

Data Index: 139574 (Open Web Text)

Max Activating Token Index: 883

Text #14

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6374. Min Act: -0.1700

Data Index: 6150178 (Open Web Text)

Max Activating Token Index: 1002

Text #15

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6085. Min Act: -0.1700

Data Index: 2000310 (Open Web Text)

Max Activating Token Index: 280

Text #16

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.5951. Min Act: -0.1700

Data Index: 2666022 (Open Web Text)

Max Activating Token Index: 196

Text #17

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6116. Min Act: -0.1700

Data Index: 6904181 (Open Web Text)

Max Activating Token Index: 222

Text #18

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.5969. Min Act: -0.1700

Data Index: 692742 (Open Web Text)

Max Activating Token Index: 194

Text #19

Max Range: 3.8666. Min Range: -3.8666

Max Act: 3.6023. Min Act: -0.1700

Data Index: 317261 (Open Web Text)

Max Activating Token Index: 631