Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 228 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.5062. Min Act: -0.1700

Data Index: 3104978 (Open Web Text)

Max Activating Token Index: 853

Text #1

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.1432. Min Act: -0.1700

Data Index: 1523614 (Open Web Text)

Max Activating Token Index: 365

Text #2

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0979. Min Act: -0.1700

Data Index: 7617624 (Open Web Text)

Max Activating Token Index: 64

Text #3

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0662. Min Act: -0.1700

Data Index: 6778820 (Open Web Text)

Max Activating Token Index: 952

Text #4

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0571. Min Act: -0.1700

Data Index: 7204103 (Open Web Text)

Max Activating Token Index: 401

Text #5

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0498. Min Act: -0.1700

Data Index: 1510610 (Open Web Text)

Max Activating Token Index: 761

Text #6

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0193. Min Act: -0.1700

Data Index: 7148279 (Open Web Text)

Max Activating Token Index: 957

Text #7

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0289. Min Act: -0.1700

Data Index: 1602257 (Open Web Text)

Max Activating Token Index: 330

Text #8

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9661. Min Act: -0.1700

Data Index: 8719081 (Open Web Text)

Max Activating Token Index: 390

Text #9

Max Range: 5.5062. Min Range: -5.5062

Max Act: 5.0016. Min Act: -0.1700

Data Index: 3236915 (Open Web Text)

Max Activating Token Index: 919

Text #10

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9838. Min Act: -0.1700

Data Index: 7945688 (Open Web Text)

Max Activating Token Index: 153

Text #11

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9729. Min Act: -0.1700

Data Index: 1066317 (Open Web Text)

Max Activating Token Index: 634

Text #12

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9762. Min Act: -0.1700

Data Index: 5613695 (Open Web Text)

Max Activating Token Index: 21

Text #13

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9587. Min Act: -0.1700

Data Index: 681684 (Open Web Text)

Max Activating Token Index: 526

Text #14

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9566. Min Act: -0.1700

Data Index: 6654053 (Open Web Text)

Max Activating Token Index: 601

Text #15

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9519. Min Act: -0.1700

Data Index: 2580009 (Open Web Text)

Max Activating Token Index: 35

Text #16

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9374. Min Act: -0.1700

Data Index: 730637 (Open Web Text)

Max Activating Token Index: 984

Text #17

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9093. Min Act: -0.1700

Data Index: 5187773 (Open Web Text)

Max Activating Token Index: 657

Text #18

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.9220. Min Act: -0.1700

Data Index: 7140623 (Open Web Text)

Max Activating Token Index: 846

Text #19

Max Range: 5.5062. Min Range: -5.5062

Max Act: 4.8495. Min Act: -0.1700

Data Index: 3444463 (Open Web Text)

Max Activating Token Index: 636