Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1848 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.3866. Min Act: -0.1700

Data Index: 4053841 (Open Web Text)

Max Activating Token Index: 557

Text #1

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.2680. Min Act: -0.1700

Data Index: 2109378 (Open Web Text)

Max Activating Token Index: 204

Text #2

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.2685. Min Act: -0.1700

Data Index: 2700306 (Open Web Text)

Max Activating Token Index: 957

Text #3

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.2266. Min Act: -0.1700

Data Index: 7451371 (Open Web Text)

Max Activating Token Index: 567

Text #4

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.1914. Min Act: -0.1700

Data Index: 2089772 (Open Web Text)

Max Activating Token Index: 190

Text #5

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.1961. Min Act: -0.1700

Data Index: 2109381 (Open Web Text)

Max Activating Token Index: 83

Text #6

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.1376. Min Act: -0.1700

Data Index: 1691202 (Open Web Text)

Max Activating Token Index: 1008

Text #7

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.1029. Min Act: -0.1700

Data Index: 4018898 (Open Web Text)

Max Activating Token Index: 491

Text #8

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.1129. Min Act: -0.1700

Data Index: 993549 (Open Web Text)

Max Activating Token Index: 547

Text #9

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0848. Min Act: -0.1700

Data Index: 955348 (Open Web Text)

Max Activating Token Index: 975

Text #10

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0836. Min Act: -0.1700

Data Index: 7149582 (Open Web Text)

Max Activating Token Index: 343

Text #11

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0786. Min Act: -0.1700

Data Index: 5620840 (Open Web Text)

Max Activating Token Index: 739

Text #12

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0432. Min Act: -0.1700

Data Index: 5456074 (Open Web Text)

Max Activating Token Index: 175

Text #13

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0854. Min Act: -0.1700

Data Index: 6984787 (Open Web Text)

Max Activating Token Index: 115

Text #14

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0867. Min Act: -0.1700

Data Index: 8812458 (Open Web Text)

Max Activating Token Index: 387

Text #15

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0608. Min Act: -0.1700

Data Index: 5132006 (Open Web Text)

Max Activating Token Index: 60

Text #16

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0424. Min Act: -0.1700

Data Index: 3555135 (Open Web Text)

Max Activating Token Index: 895

Text #17

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0473. Min Act: -0.1700

Data Index: 2233836 (Open Web Text)

Max Activating Token Index: 672

Text #18

Max Range: 5.3866. Min Range: -5.3866

Max Act: 5.0063. Min Act: -0.1700

Data Index: 1098631 (Open Web Text)

Max Activating Token Index: 747

Text #19

Max Range: 5.3866. Min Range: -5.3866

Max Act: 4.9867. Min Act: -0.1700

Data Index: 1381520 (Open Web Text)

Max Activating Token Index: 267