Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2566 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.7488. Min Act: -0.1700

Data Index: 2734943 (Open Web Text)

Max Activating Token Index: 759

Text #1

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.6436. Min Act: -0.1700

Data Index: 5840522 (Open Web Text)

Max Activating Token Index: 654

Text #2

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.6312. Min Act: -0.1700

Data Index: 1444849 (Open Web Text)

Max Activating Token Index: 127

Text #3

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.6293. Min Act: -0.1700

Data Index: 1958455 (Open Web Text)

Max Activating Token Index: 130

Text #4

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5987. Min Act: -0.1700

Data Index: 2975388 (Open Web Text)

Max Activating Token Index: 465

Text #5

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.6129. Min Act: -0.1700

Data Index: 6511200 (Open Web Text)

Max Activating Token Index: 101

Text #6

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.6074. Min Act: -0.1700

Data Index: 7500871 (Open Web Text)

Max Activating Token Index: 370

Text #7

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5827. Min Act: -0.1700

Data Index: 2188390 (Open Web Text)

Max Activating Token Index: 942

Text #8

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5847. Min Act: -0.1700

Data Index: 7059492 (Open Web Text)

Max Activating Token Index: 756

Text #9

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5609. Min Act: -0.1700

Data Index: 6678525 (Open Web Text)

Max Activating Token Index: 578

Text #10

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5952. Min Act: -0.1700

Data Index: 3111437 (Open Web Text)

Max Activating Token Index: 999

Text #11

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5625. Min Act: -0.1700

Data Index: 6601872 (Open Web Text)

Max Activating Token Index: 401

Text #12

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5307. Min Act: -0.1700

Data Index: 258200 (Open Web Text)

Max Activating Token Index: 160

Text #13

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5599. Min Act: -0.1700

Data Index: 6558325 (Open Web Text)

Max Activating Token Index: 239

Text #14

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5692. Min Act: -0.1700

Data Index: 6191150 (Open Web Text)

Max Activating Token Index: 817

Text #15

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5742. Min Act: -0.1700

Data Index: 5531175 (Open Web Text)

Max Activating Token Index: 158

Text #16

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5952. Min Act: -0.1700

Data Index: 5195788 (Open Web Text)

Max Activating Token Index: 88

Text #17

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5703. Min Act: -0.1700

Data Index: 4126488 (Open Web Text)

Max Activating Token Index: 500

Text #18

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.5671. Min Act: -0.1700

Data Index: 2504177 (Open Web Text)

Max Activating Token Index: 448

Text #19

Max Range: 3.7488. Min Range: -3.7488

Max Act: 3.6043. Min Act: -0.1700

Data Index: 1064499 (Open Web Text)

Max Activating Token Index: 100