Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3062 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.8810. Min Act: -0.1700

Data Index: 922842 (Open Web Text)

Max Activating Token Index: 303

Text #1

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.8784. Min Act: -0.1700

Data Index: 3863190 (Open Web Text)

Max Activating Token Index: 717

Text #2

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.8183. Min Act: -0.1700

Data Index: 1366061 (Open Web Text)

Max Activating Token Index: 517

Text #3

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.8150. Min Act: -0.1700

Data Index: 1557359 (Open Web Text)

Max Activating Token Index: 549

Text #4

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7763. Min Act: -0.1700

Data Index: 2144237 (Open Web Text)

Max Activating Token Index: 13

Text #5

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7822. Min Act: -0.1700

Data Index: 6590587 (Open Web Text)

Max Activating Token Index: 924

Text #6

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7553. Min Act: -0.1700

Data Index: 5228359 (Open Web Text)

Max Activating Token Index: 808

Text #7

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7510. Min Act: -0.1700

Data Index: 3406277 (Open Web Text)

Max Activating Token Index: 592

Text #8

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6882. Min Act: -0.1700

Data Index: 5877900 (Open Web Text)

Max Activating Token Index: 881

Text #9

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7109. Min Act: -0.1700

Data Index: 5238569 (Open Web Text)

Max Activating Token Index: 452

Text #10

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7120. Min Act: -0.1700

Data Index: 3954562 (Open Web Text)

Max Activating Token Index: 605

Text #11

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6998. Min Act: -0.1700

Data Index: 465437 (Open Web Text)

Max Activating Token Index: 14

Text #12

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.7163. Min Act: -0.1700

Data Index: 1053813 (Open Web Text)

Max Activating Token Index: 454

Text #13

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6742. Min Act: -0.1700

Data Index: 4969354 (Open Web Text)

Max Activating Token Index: 530

Text #14

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6676. Min Act: -0.1700

Data Index: 2899985 (Open Web Text)

Max Activating Token Index: 286

Text #15

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6695. Min Act: -0.1700

Data Index: 2245750 (Open Web Text)

Max Activating Token Index: 669

Text #16

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6844. Min Act: -0.1700

Data Index: 3812896 (Open Web Text)

Max Activating Token Index: 23

Text #17

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6827. Min Act: -0.1700

Data Index: 1118703 (Open Web Text)

Max Activating Token Index: 384

Text #18

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6704. Min Act: -0.1700

Data Index: 1650851 (Open Web Text)

Max Activating Token Index: 610

Text #19

Max Range: 3.8810. Min Range: -3.8810

Max Act: 3.6751. Min Act: -0.1700

Data Index: 5375186 (Open Web Text)

Max Activating Token Index: 871