Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1156 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.3492. Min Act: -0.1700

Data Index: 3072757 (Open Web Text)

Max Activating Token Index: 879

Text #1

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1918. Min Act: -0.1700

Data Index: 7223480 (Open Web Text)

Max Activating Token Index: 464

Text #2

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1853. Min Act: -0.1700

Data Index: 5481934 (Open Web Text)

Max Activating Token Index: 519

Text #3

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1502. Min Act: -0.1700

Data Index: 3414041 (Open Web Text)

Max Activating Token Index: 341

Text #4

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1258. Min Act: -0.1700

Data Index: 1833486 (Open Web Text)

Max Activating Token Index: 617

Text #5

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1377. Min Act: -0.1700

Data Index: 5134300 (Open Web Text)

Max Activating Token Index: 84

Text #6

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1071. Min Act: -0.1700

Data Index: 2174964 (Open Web Text)

Max Activating Token Index: 427

Text #7

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1096. Min Act: -0.1700

Data Index: 838635 (Open Web Text)

Max Activating Token Index: 371

Text #8

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1007. Min Act: -0.1700

Data Index: 5254120 (Open Web Text)

Max Activating Token Index: 83

Text #9

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1110. Min Act: -0.1700

Data Index: 1965284 (Open Web Text)

Max Activating Token Index: 915

Text #10

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0943. Min Act: -0.1700

Data Index: 3414215 (Open Web Text)

Max Activating Token Index: 37

Text #11

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1000. Min Act: -0.1700

Data Index: 2071486 (Open Web Text)

Max Activating Token Index: 705

Text #12

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0870. Min Act: -0.1700

Data Index: 5706695 (Open Web Text)

Max Activating Token Index: 914

Text #13

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1044. Min Act: -0.1700

Data Index: 3115416 (Open Web Text)

Max Activating Token Index: 165

Text #14

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1078. Min Act: -0.1700

Data Index: 5182739 (Open Web Text)

Max Activating Token Index: 707

Text #15

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0868. Min Act: -0.1700

Data Index: 5650163 (Open Web Text)

Max Activating Token Index: 49

Text #16

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0866. Min Act: -0.1700

Data Index: 1797427 (Open Web Text)

Max Activating Token Index: 701

Text #17

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0734. Min Act: -0.1700

Data Index: 6347640 (Open Web Text)

Max Activating Token Index: 335

Text #18

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0866. Min Act: -0.1700

Data Index: 8765113 (Open Web Text)

Max Activating Token Index: 470

Text #19

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0855. Min Act: -0.1700

Data Index: 2335792 (Open Web Text)

Max Activating Token Index: 775