Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2039 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.3229. Min Range: -6.3229

Max Act: 6.3229. Min Act: -0.1700

Data Index: 7860272 (Open Web Text)

Max Activating Token Index: 146

Text #1

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.9916. Min Act: -0.1700

Data Index: 4285833 (Open Web Text)

Max Activating Token Index: 224

Text #2

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.9537. Min Act: -0.1700

Data Index: 1821436 (Open Web Text)

Max Activating Token Index: 331

Text #3

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.6605. Min Act: -0.1700

Data Index: 41197 (Open Web Text)

Max Activating Token Index: 817

Text #4

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.5180. Min Act: -0.1700

Data Index: 7851393 (Open Web Text)

Max Activating Token Index: 296

Text #5

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.5181. Min Act: -0.1700

Data Index: 4211666 (Open Web Text)

Max Activating Token Index: 105

Text #6

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.4495. Min Act: -0.1700

Data Index: 6150460 (Open Web Text)

Max Activating Token Index: 578

Text #7

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.4533. Min Act: -0.1700

Data Index: 2489886 (Open Web Text)

Max Activating Token Index: 344

Text #8

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.4559. Min Act: -0.1700

Data Index: 8386835 (Open Web Text)

Max Activating Token Index: 321

Text #9

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.3856. Min Act: -0.1700

Data Index: 7605965 (Open Web Text)

Max Activating Token Index: 941

Text #10

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.2988. Min Act: -0.1700

Data Index: 6182633 (Open Web Text)

Max Activating Token Index: 895

Text #11

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.2906. Min Act: -0.1700

Data Index: 6886836 (Open Web Text)

Max Activating Token Index: 392

Text #12

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.3018. Min Act: -0.1700

Data Index: 6900122 (Open Web Text)

Max Activating Token Index: 878

Text #13

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.3316. Min Act: -0.1700

Data Index: 6150470 (Open Web Text)

Max Activating Token Index: 510

Text #14

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.3470. Min Act: -0.1700

Data Index: 4630209 (Open Web Text)

Max Activating Token Index: 451

Text #15

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.2967. Min Act: -0.1700

Data Index: 1584309 (Open Web Text)

Max Activating Token Index: 121

Text #16

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.2817. Min Act: -0.1700

Data Index: 4284073 (Open Web Text)

Max Activating Token Index: 124

Text #17

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.2584. Min Act: -0.1700

Data Index: 8826332 (Open Web Text)

Max Activating Token Index: 651

Text #18

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.2739. Min Act: -0.1700

Data Index: 973320 (Open Web Text)

Max Activating Token Index: 456

Text #19

Max Range: 6.3229. Min Range: -6.3229

Max Act: 5.1552. Min Act: -0.1700

Data Index: 6557282 (Open Web Text)

Max Activating Token Index: 840