Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1060 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7713. Min Act: -0.1680

Data Index: 8170957 (Open Web Text)

Max Activating Token Index: 265

Text #1

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7720. Min Act: -0.1700

Data Index: 4261821 (Open Web Text)

Max Activating Token Index: 450

Text #2

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7629. Min Act: -0.1700

Data Index: 284624 (Open Web Text)

Max Activating Token Index: 991

Text #3

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7648. Min Act: -0.1700

Data Index: 58737 (Open Web Text)

Max Activating Token Index: 512

Text #4

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7292. Min Act: -0.1700

Data Index: 1697516 (Open Web Text)

Max Activating Token Index: 169

Text #5

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7279. Min Act: -0.1698

Data Index: 1089042 (Open Web Text)

Max Activating Token Index: 723

Text #6

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7285. Min Act: -0.1700

Data Index: 4086062 (Open Web Text)

Max Activating Token Index: 914

Text #7

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7254. Min Act: -0.1700

Data Index: 1322097 (Open Web Text)

Max Activating Token Index: 445

Text #8

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7310. Min Act: -0.1700

Data Index: 5776193 (Open Web Text)

Max Activating Token Index: 998

Text #9

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7134. Min Act: -0.1700

Data Index: 2531292 (Open Web Text)

Max Activating Token Index: 591

Text #10

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7156. Min Act: -0.1700

Data Index: 1162102 (Open Web Text)

Max Activating Token Index: 643

Text #11

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7139. Min Act: -0.1700

Data Index: 2839149 (Open Web Text)

Max Activating Token Index: 486

Text #12

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7141. Min Act: -0.1700

Data Index: 4367262 (Open Web Text)

Max Activating Token Index: 562

Text #13

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7017. Min Act: -0.1700

Data Index: 2871137 (Open Web Text)

Max Activating Token Index: 508

Text #14

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7007. Min Act: -0.1700

Data Index: 5997763 (Open Web Text)

Max Activating Token Index: 941

Text #15

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7012. Min Act: -0.1700

Data Index: 8326479 (Open Web Text)

Max Activating Token Index: 339

Text #16

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.6969. Min Act: -0.1690

Data Index: 1027978 (Open Web Text)

Max Activating Token Index: 748

Text #17

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.6901. Min Act: -0.1700

Data Index: 3076646 (Open Web Text)

Max Activating Token Index: 398

Text #18

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.7028. Min Act: -0.1700

Data Index: 6857795 (Open Web Text)

Max Activating Token Index: 501

Text #19

Max Range: 1.7720. Min Range: -1.7720

Max Act: 1.6941. Min Act: -0.1700

Data Index: 7317275 (Open Web Text)

Max Activating Token Index: 355