Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 408 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.1012. Min Act: -0.1700

Data Index: 283185 (Open Web Text)

Max Activating Token Index: 563

Text #1

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0597. Min Act: -0.1700

Data Index: 6932766 (Open Web Text)

Max Activating Token Index: 118

Text #2

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0421. Min Act: -0.1700

Data Index: 2824023 (Open Web Text)

Max Activating Token Index: 997

Text #3

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0539. Min Act: -0.1700

Data Index: 1306554 (Open Web Text)

Max Activating Token Index: 588

Text #4

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0579. Min Act: -0.1700

Data Index: 1306555 (Open Web Text)

Max Activating Token Index: 430

Text #5

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0239. Min Act: -0.1700

Data Index: 629976 (Open Web Text)

Max Activating Token Index: 639

Text #6

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0220. Min Act: -0.1700

Data Index: 608364 (Open Web Text)

Max Activating Token Index: 610

Text #7

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0132. Min Act: -0.1700

Data Index: 5059194 (Open Web Text)

Max Activating Token Index: 777

Text #8

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9937. Min Act: -0.1700

Data Index: 5955605 (Open Web Text)

Max Activating Token Index: 500

Text #9

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0038. Min Act: -0.1700

Data Index: 3699018 (Open Web Text)

Max Activating Token Index: 49

Text #10

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0101. Min Act: -0.1700

Data Index: 4369744 (Open Web Text)

Max Activating Token Index: 624

Text #11

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9931. Min Act: -0.1700

Data Index: 1009873 (Open Web Text)

Max Activating Token Index: 841

Text #12

Max Range: 3.1012. Min Range: -3.1012

Max Act: 3.0014. Min Act: -0.1700

Data Index: 6115682 (Open Web Text)

Max Activating Token Index: 925

Text #13

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9637. Min Act: -0.1700

Data Index: 4561463 (Open Web Text)

Max Activating Token Index: 109

Text #14

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9848. Min Act: -0.1700

Data Index: 6311584 (Open Web Text)

Max Activating Token Index: 253

Text #15

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9816. Min Act: -0.1700

Data Index: 5073375 (Open Web Text)

Max Activating Token Index: 76

Text #16

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9692. Min Act: -0.1700

Data Index: 2215845 (Open Web Text)

Max Activating Token Index: 26

Text #17

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9774. Min Act: -0.1700

Data Index: 7188971 (Open Web Text)

Max Activating Token Index: 852

Text #18

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9496. Min Act: -0.1700

Data Index: 5759189 (Open Web Text)

Max Activating Token Index: 879

Text #19

Max Range: 3.1012. Min Range: -3.1012

Max Act: 2.9820. Min Act: -0.1700

Data Index: 5273062 (Open Web Text)

Max Activating Token Index: 755