Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1656 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.3940. Min Act: -0.1700

Data Index: 343012 (Open Web Text)

Max Activating Token Index: 952

Text #1

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.3158. Min Act: -0.1700

Data Index: 7899048 (Open Web Text)

Max Activating Token Index: 37

Text #2

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2827. Min Act: -0.1700

Data Index: 6792162 (Open Web Text)

Max Activating Token Index: 593

Text #3

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2884. Min Act: -0.1700

Data Index: 3825606 (Open Web Text)

Max Activating Token Index: 65

Text #4

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2520. Min Act: -0.1700

Data Index: 4209985 (Open Web Text)

Max Activating Token Index: 760

Text #5

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2648. Min Act: -0.1700

Data Index: 4214749 (Open Web Text)

Max Activating Token Index: 699

Text #6

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2037. Min Act: -0.1700

Data Index: 3294016 (Open Web Text)

Max Activating Token Index: 50

Text #7

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2333. Min Act: -0.1700

Data Index: 6378585 (Open Web Text)

Max Activating Token Index: 1013

Text #8

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2377. Min Act: -0.1700

Data Index: 765885 (Open Web Text)

Max Activating Token Index: 699

Text #9

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2159. Min Act: -0.1700

Data Index: 1161669 (Open Web Text)

Max Activating Token Index: 617

Text #10

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2101. Min Act: -0.1700

Data Index: 4994175 (Open Web Text)

Max Activating Token Index: 574

Text #11

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1948. Min Act: -0.1700

Data Index: 8403320 (Open Web Text)

Max Activating Token Index: 429

Text #12

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1376. Min Act: -0.1700

Data Index: 2088673 (Open Web Text)

Max Activating Token Index: 809

Text #13

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1806. Min Act: -0.1700

Data Index: 1621589 (Open Web Text)

Max Activating Token Index: 299

Text #14

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.2113. Min Act: -0.1700

Data Index: 832376 (Open Web Text)

Max Activating Token Index: 228

Text #15

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1654. Min Act: -0.1700

Data Index: 3171352 (Open Web Text)

Max Activating Token Index: 282

Text #16

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1356. Min Act: -0.1700

Data Index: 3124368 (Open Web Text)

Max Activating Token Index: 512

Text #17

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1472. Min Act: -0.1700

Data Index: 969326 (Open Web Text)

Max Activating Token Index: 320

Text #18

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1774. Min Act: -0.1700

Data Index: 7850288 (Open Web Text)

Max Activating Token Index: 75

Text #19

Max Range: 4.3940. Min Range: -4.3940

Max Act: 4.1336. Min Act: -0.1700

Data Index: 4680279 (Open Web Text)

Max Activating Token Index: 229