Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 58 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.8918. Min Act: -0.1700

Data Index: 6446 (Open Web Text)

Max Activating Token Index: 972

Text #1

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.8811. Min Act: -0.1700

Data Index: 756689 (Open Web Text)

Max Activating Token Index: 113

Text #2

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.8702. Min Act: -0.1700

Data Index: 5894151 (Open Web Text)

Max Activating Token Index: 382

Text #3

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.8264. Min Act: -0.1700

Data Index: 1325754 (Open Web Text)

Max Activating Token Index: 826

Text #4

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7874. Min Act: -0.1700

Data Index: 5220360 (Open Web Text)

Max Activating Token Index: 113

Text #5

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.8069. Min Act: -0.1700

Data Index: 717912 (Open Web Text)

Max Activating Token Index: 621

Text #6

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7994. Min Act: -0.1700

Data Index: 4191534 (Open Web Text)

Max Activating Token Index: 133

Text #7

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7806. Min Act: -0.1700

Data Index: 2285205 (Open Web Text)

Max Activating Token Index: 265

Text #8

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7727. Min Act: -0.1700

Data Index: 6735636 (Open Web Text)

Max Activating Token Index: 308

Text #9

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7548. Min Act: -0.1700

Data Index: 4953568 (Open Web Text)

Max Activating Token Index: 906

Text #10

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7384. Min Act: -0.1700

Data Index: 556407 (Open Web Text)

Max Activating Token Index: 804

Text #11

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7467. Min Act: -0.1700

Data Index: 1637452 (Open Web Text)

Max Activating Token Index: 782

Text #12

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7468. Min Act: -0.1700

Data Index: 1694454 (Open Web Text)

Max Activating Token Index: 794

Text #13

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7212. Min Act: -0.1700

Data Index: 8736119 (Open Web Text)

Max Activating Token Index: 573

Text #14

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7184. Min Act: -0.1700

Data Index: 1459448 (Open Web Text)

Max Activating Token Index: 549

Text #15

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7396. Min Act: -0.1700

Data Index: 7764391 (Open Web Text)

Max Activating Token Index: 925

Text #16

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7119. Min Act: -0.1700

Data Index: 7239410 (Open Web Text)

Max Activating Token Index: 883

Text #17

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7276. Min Act: -0.1700

Data Index: 7172043 (Open Web Text)

Max Activating Token Index: 366

Text #18

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.7096. Min Act: -0.1700

Data Index: 324335 (Open Web Text)

Max Activating Token Index: 741

Text #19

Max Range: 3.8918. Min Range: -3.8918

Max Act: 3.6786. Min Act: -0.1700

Data Index: 7049386 (Open Web Text)

Max Activating Token Index: 386