Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 150 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.9132. Min Act: -0.1700

Data Index: 5753144 (Open Web Text)

Max Activating Token Index: 998

Text #1

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.9004. Min Act: -0.1700

Data Index: 1007342 (Open Web Text)

Max Activating Token Index: 662

Text #2

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8674. Min Act: -0.1700

Data Index: 2167362 (Open Web Text)

Max Activating Token Index: 1004

Text #3

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8509. Min Act: -0.1700

Data Index: 8740841 (Open Web Text)

Max Activating Token Index: 251

Text #4

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8488. Min Act: -0.1700

Data Index: 6258382 (Open Web Text)

Max Activating Token Index: 268

Text #5

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8472. Min Act: -0.1700

Data Index: 8499371 (Open Web Text)

Max Activating Token Index: 33

Text #6

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8415. Min Act: -0.1700

Data Index: 7274154 (Open Web Text)

Max Activating Token Index: 1005

Text #7

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8430. Min Act: -0.1700

Data Index: 2704268 (Open Web Text)

Max Activating Token Index: 867

Text #8

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8364. Min Act: -0.1700

Data Index: 7382744 (Open Web Text)

Max Activating Token Index: 24

Text #9

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8350. Min Act: -0.1700

Data Index: 6361769 (Open Web Text)

Max Activating Token Index: 601

Text #10

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8302. Min Act: -0.1700

Data Index: 1676031 (Open Web Text)

Max Activating Token Index: 8

Text #11

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8329. Min Act: -0.1700

Data Index: 197701 (Open Web Text)

Max Activating Token Index: 689

Text #12

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8330. Min Act: -0.1700

Data Index: 8039026 (Open Web Text)

Max Activating Token Index: 90

Text #13

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8318. Min Act: -0.1700

Data Index: 4902489 (Open Web Text)

Max Activating Token Index: 491

Text #14

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8220. Min Act: -0.1700

Data Index: 4242212 (Open Web Text)

Max Activating Token Index: 463

Text #15

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8127. Min Act: -0.1700

Data Index: 1541355 (Open Web Text)

Max Activating Token Index: 918

Text #16

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8176. Min Act: -0.1700

Data Index: 6032646 (Open Web Text)

Max Activating Token Index: 836

Text #17

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8212. Min Act: -0.1700

Data Index: 4657602 (Open Web Text)

Max Activating Token Index: 87

Text #18

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8163. Min Act: -0.1700

Data Index: 7248610 (Open Web Text)

Max Activating Token Index: 975

Text #19

Max Range: 1.9132. Min Range: -1.9132

Max Act: 1.8207. Min Act: -0.1700

Data Index: 1952392 (Open Web Text)

Max Activating Token Index: 843