Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 300 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 9.3025. Min Range: -9.3025

Max Act: 9.3025. Min Act: -0.1689

Data Index: 2456312 (Open Web Text)

Max Activating Token Index: 541

Text #1

Max Range: 9.3025. Min Range: -9.3025

Max Act: 9.2505. Min Act: -0.1700

Data Index: 2456309 (Open Web Text)

Max Activating Token Index: 302

Text #2

Max Range: 9.3025. Min Range: -9.3025

Max Act: 9.2290. Min Act: -0.1700

Data Index: 65064 (Open Web Text)

Max Activating Token Index: 838

Text #3

Max Range: 9.3025. Min Range: -9.3025

Max Act: 9.1169. Min Act: -0.1148

Data Index: 65067 (Open Web Text)

Max Activating Token Index: 456

Text #4

Max Range: 9.3025. Min Range: -9.3025

Max Act: 9.0690. Min Act: -0.0524

Data Index: 65066 (Open Web Text)

Max Activating Token Index: 475

Text #5

Max Range: 9.3025. Min Range: -9.3025

Max Act: 9.0433. Min Act: -0.1700

Data Index: 6002547 (Open Web Text)

Max Activating Token Index: 860

Text #6

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.9193. Min Act: -0.1700

Data Index: 1169836 (Open Web Text)

Max Activating Token Index: 21

Text #7

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.9157. Min Act: -0.1650

Data Index: 65068 (Open Web Text)

Max Activating Token Index: 458

Text #8

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.8445. Min Act: -0.1698

Data Index: 2456311 (Open Web Text)

Max Activating Token Index: 615

Text #9

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.8447. Min Act: -0.1681

Data Index: 65069 (Open Web Text)

Max Activating Token Index: 522

Text #10

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.8241. Min Act: -0.1693

Data Index: 2456314 (Open Web Text)

Max Activating Token Index: 29

Text #11

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.7478. Min Act: -0.1697

Data Index: 2456313 (Open Web Text)

Max Activating Token Index: 592

Text #12

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.7355. Min Act: -0.1700

Data Index: 5845977 (Open Web Text)

Max Activating Token Index: 531

Text #13

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.7003. Min Act: -0.1700

Data Index: 2655696 (Open Web Text)

Max Activating Token Index: 856

Text #14

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.6687. Min Act: -0.1700

Data Index: 1237189 (Open Web Text)

Max Activating Token Index: 25

Text #15

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.6149. Min Act: -0.1700

Data Index: 3544102 (Open Web Text)

Max Activating Token Index: 62

Text #16

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.5688. Min Act: -0.1700

Data Index: 3759917 (Open Web Text)

Max Activating Token Index: 13

Text #17

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.4726. Min Act: -0.1700

Data Index: 2459276 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.3868. Min Act: -0.1615

Data Index: 65065 (Open Web Text)

Max Activating Token Index: 45

Text #19

Max Range: 9.3025. Min Range: -9.3025

Max Act: 8.3458. Min Act: -0.1700

Data Index: 2027436 (Open Web Text)

Max Activating Token Index: 565