Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1318 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.5709. Min Act: -0.1700

Data Index: 394545 (Open Web Text)

Max Activating Token Index: 18

Text #1

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.4062. Min Act: -0.1700

Data Index: 4918901 (Open Web Text)

Max Activating Token Index: 423

Text #2

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.3142. Min Act: -0.1700

Data Index: 3977347 (Open Web Text)

Max Activating Token Index: 29

Text #3

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.3102. Min Act: -0.1700

Data Index: 8793302 (Open Web Text)

Max Activating Token Index: 71

Text #4

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.2753. Min Act: -0.1700

Data Index: 4673750 (Open Web Text)

Max Activating Token Index: 638

Text #5

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.2008. Min Act: -0.1700

Data Index: 3823095 (Open Web Text)

Max Activating Token Index: 334

Text #6

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.2141. Min Act: -0.1700

Data Index: 5944152 (Open Web Text)

Max Activating Token Index: 646

Text #7

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1608. Min Act: -0.1700

Data Index: 8716475 (Open Web Text)

Max Activating Token Index: 26

Text #8

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1755. Min Act: -0.1700

Data Index: 181516 (Open Web Text)

Max Activating Token Index: 186

Text #9

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1758. Min Act: -0.1700

Data Index: 6664173 (Open Web Text)

Max Activating Token Index: 164

Text #10

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1198. Min Act: -0.1700

Data Index: 7651411 (Open Web Text)

Max Activating Token Index: 499

Text #11

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1469. Min Act: -0.1700

Data Index: 2210934 (Open Web Text)

Max Activating Token Index: 115

Text #12

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1295. Min Act: -0.1700

Data Index: 7203152 (Open Web Text)

Max Activating Token Index: 88

Text #13

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1371. Min Act: -0.1700

Data Index: 902684 (Open Web Text)

Max Activating Token Index: 65

Text #14

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1166. Min Act: -0.1700

Data Index: 8514857 (Open Web Text)

Max Activating Token Index: 395

Text #15

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1104. Min Act: -0.1700

Data Index: 573628 (Open Web Text)

Max Activating Token Index: 218

Text #16

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.1144. Min Act: -0.1700

Data Index: 798684 (Open Web Text)

Max Activating Token Index: 111

Text #17

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.0840. Min Act: -0.1700

Data Index: 5712119 (Open Web Text)

Max Activating Token Index: 753

Text #18

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.0849. Min Act: -0.1700

Data Index: 7057532 (Open Web Text)

Max Activating Token Index: 518

Text #19

Max Range: 3.5709. Min Range: -3.5709

Max Act: 3.0750. Min Act: -0.1700

Data Index: 5291818 (Open Web Text)

Max Activating Token Index: 515