Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2805 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.3140. Min Act: -0.1700

Data Index: 3525625 (Open Web Text)

Max Activating Token Index: 380

Text #1

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.3136. Min Act: -0.1700

Data Index: 3586875 (Open Web Text)

Max Activating Token Index: 108

Text #2

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.2789. Min Act: -0.1700

Data Index: 7604961 (Open Web Text)

Max Activating Token Index: 389

Text #3

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.2037. Min Act: -0.1700

Data Index: 8109199 (Open Web Text)

Max Activating Token Index: 665

Text #4

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1547. Min Act: -0.1700

Data Index: 976400 (Open Web Text)

Max Activating Token Index: 709

Text #5

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1719. Min Act: -0.1700

Data Index: 8252033 (Open Web Text)

Max Activating Token Index: 79

Text #6

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1333. Min Act: -0.1700

Data Index: 7799777 (Open Web Text)

Max Activating Token Index: 74

Text #7

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1032. Min Act: -0.1700

Data Index: 6492368 (Open Web Text)

Max Activating Token Index: 170

Text #8

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1640. Min Act: -0.1700

Data Index: 6454400 (Open Web Text)

Max Activating Token Index: 694

Text #9

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1183. Min Act: -0.1700

Data Index: 7340602 (Open Web Text)

Max Activating Token Index: 488

Text #10

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1355. Min Act: -0.1700

Data Index: 1059863 (Open Web Text)

Max Activating Token Index: 1003

Text #11

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.0528. Min Act: -0.1700

Data Index: 1442992 (Open Web Text)

Max Activating Token Index: 593

Text #12

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1162. Min Act: -0.1700

Data Index: 7965423 (Open Web Text)

Max Activating Token Index: 411

Text #13

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1126. Min Act: -0.1700

Data Index: 2837418 (Open Web Text)

Max Activating Token Index: 334

Text #14

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.0833. Min Act: -0.1700

Data Index: 2741150 (Open Web Text)

Max Activating Token Index: 63

Text #15

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.0913. Min Act: -0.1700

Data Index: 4461546 (Open Web Text)

Max Activating Token Index: 19

Text #16

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.1324. Min Act: -0.1700

Data Index: 3984072 (Open Web Text)

Max Activating Token Index: 577

Text #17

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.0580. Min Act: -0.1700

Data Index: 5340691 (Open Web Text)

Max Activating Token Index: 933

Text #18

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.0481. Min Act: -0.1700

Data Index: 2789808 (Open Web Text)

Max Activating Token Index: 59

Text #19

Max Range: 5.3140. Min Range: -5.3140

Max Act: 5.0431. Min Act: -0.1700

Data Index: 6750841 (Open Web Text)

Max Activating Token Index: 675