Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1670 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.1622. Min Act: -0.1281

Data Index: 6935081 (Open Web Text)

Max Activating Token Index: 59

Text #1

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0976. Min Act: -0.1700

Data Index: 6208143 (Open Web Text)

Max Activating Token Index: 51

Text #2

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0721. Min Act: -0.1700

Data Index: 850203 (Open Web Text)

Max Activating Token Index: 58

Text #3

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0838. Min Act: -0.1700

Data Index: 2160827 (Open Web Text)

Max Activating Token Index: 285

Text #4

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0629. Min Act: -0.1700

Data Index: 6844046 (Open Web Text)

Max Activating Token Index: 77

Text #5

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0167. Min Act: -0.1700

Data Index: 8271841 (Open Web Text)

Max Activating Token Index: 22

Text #6

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0127. Min Act: -0.1697

Data Index: 3953841 (Open Web Text)

Max Activating Token Index: 69

Text #7

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0244. Min Act: -0.1552

Data Index: 1656595 (Open Web Text)

Max Activating Token Index: 193

Text #8

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0025. Min Act: -0.1700

Data Index: 5400566 (Open Web Text)

Max Activating Token Index: 169

Text #9

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0069. Min Act: -0.1656

Data Index: 286093 (Open Web Text)

Max Activating Token Index: 739

Text #10

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9772. Min Act: -0.1700

Data Index: 3427014 (Open Web Text)

Max Activating Token Index: 286

Text #11

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9844. Min Act: -0.1700

Data Index: 1562353 (Open Web Text)

Max Activating Token Index: 425

Text #12

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9990. Min Act: -0.1699

Data Index: 286094 (Open Web Text)

Max Activating Token Index: 55

Text #13

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9754. Min Act: -0.1700

Data Index: 1791616 (Open Web Text)

Max Activating Token Index: 43

Text #14

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0053. Min Act: -0.1700

Data Index: 7762523 (Open Web Text)

Max Activating Token Index: 698

Text #15

Max Range: 4.1622. Min Range: -4.1622

Max Act: 4.0010. Min Act: -0.1700

Data Index: 399033 (Open Web Text)

Max Activating Token Index: 31

Text #16

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9892. Min Act: -0.1700

Data Index: 4048897 (Open Web Text)

Max Activating Token Index: 62

Text #17

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9622. Min Act: -0.1499

Data Index: 1356396 (Open Web Text)

Max Activating Token Index: 123

Text #18

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9458. Min Act: -0.1694

Data Index: 7209152 (Open Web Text)

Max Activating Token Index: 122

Text #19

Max Range: 4.1622. Min Range: -4.1622

Max Act: 3.9344. Min Act: -0.1700

Data Index: 7782195 (Open Web Text)

Max Activating Token Index: 13