Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1944 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.9536. Min Act: -0.1700

Data Index: 3525642 (Open Web Text)

Max Activating Token Index: 192

Text #1

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.9219. Min Act: -0.1700

Data Index: 3525640 (Open Web Text)

Max Activating Token Index: 1007

Text #2

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8819. Min Act: -0.1700

Data Index: 6826637 (Open Web Text)

Max Activating Token Index: 546

Text #3

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8645. Min Act: -0.1700

Data Index: 3850780 (Open Web Text)

Max Activating Token Index: 475

Text #4

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8534. Min Act: -0.1700

Data Index: 5541735 (Open Web Text)

Max Activating Token Index: 467

Text #5

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8187. Min Act: -0.1700

Data Index: 2520757 (Open Web Text)

Max Activating Token Index: 84

Text #6

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8052. Min Act: -0.1700

Data Index: 4456808 (Open Web Text)

Max Activating Token Index: 398

Text #7

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8123. Min Act: -0.1700

Data Index: 8601400 (Open Web Text)

Max Activating Token Index: 440

Text #8

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8105. Min Act: -0.1700

Data Index: 1761345 (Open Web Text)

Max Activating Token Index: 443

Text #9

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.8051. Min Act: -0.1700

Data Index: 3401582 (Open Web Text)

Max Activating Token Index: 682

Text #10

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7940. Min Act: -0.1700

Data Index: 61784 (Open Web Text)

Max Activating Token Index: 60

Text #11

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7921. Min Act: -0.1699

Data Index: 3582587 (Open Web Text)

Max Activating Token Index: 430

Text #12

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7829. Min Act: -0.1700

Data Index: 7107755 (Open Web Text)

Max Activating Token Index: 153

Text #13

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7862. Min Act: -0.1700

Data Index: 1852956 (Open Web Text)

Max Activating Token Index: 541

Text #14

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7900. Min Act: -0.1700

Data Index: 6561134 (Open Web Text)

Max Activating Token Index: 402

Text #15

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7720. Min Act: -0.1698

Data Index: 5608400 (Open Web Text)

Max Activating Token Index: 398

Text #16

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7813. Min Act: -0.1700

Data Index: 1695294 (Open Web Text)

Max Activating Token Index: 30

Text #17

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7614. Min Act: -0.1700

Data Index: 3152035 (Open Web Text)

Max Activating Token Index: 998

Text #18

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7703. Min Act: -0.1700

Data Index: 1233319 (Open Web Text)

Max Activating Token Index: 956

Text #19

Max Range: 1.9536. Min Range: -1.9536

Max Act: 1.7782. Min Act: -0.1700

Data Index: 5631033 (Open Web Text)

Max Activating Token Index: 644