Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1599 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.3561. Min Act: -0.1700

Data Index: 4093981 (Open Web Text)

Max Activating Token Index: 378

Text #1

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.3461. Min Act: -0.1700

Data Index: 5140955 (Open Web Text)

Max Activating Token Index: 161

Text #2

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2827. Min Act: -0.1700

Data Index: 777079 (Open Web Text)

Max Activating Token Index: 347

Text #3

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2865. Min Act: -0.1700

Data Index: 294823 (Open Web Text)

Max Activating Token Index: 152

Text #4

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2741. Min Act: -0.1700

Data Index: 6691021 (Open Web Text)

Max Activating Token Index: 220

Text #5

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2494. Min Act: -0.1700

Data Index: 1190501 (Open Web Text)

Max Activating Token Index: 62

Text #6

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2298. Min Act: -0.1700

Data Index: 6899706 (Open Web Text)

Max Activating Token Index: 28

Text #7

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2072. Min Act: -0.1700

Data Index: 6441462 (Open Web Text)

Max Activating Token Index: 275

Text #8

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2343. Min Act: -0.1700

Data Index: 8071339 (Open Web Text)

Max Activating Token Index: 146

Text #9

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2119. Min Act: -0.1700

Data Index: 7913454 (Open Web Text)

Max Activating Token Index: 731

Text #10

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2432. Min Act: -0.1700

Data Index: 5389910 (Open Web Text)

Max Activating Token Index: 726

Text #11

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2052. Min Act: -0.1700

Data Index: 5621183 (Open Web Text)

Max Activating Token Index: 182

Text #12

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2223. Min Act: -0.1700

Data Index: 7114064 (Open Web Text)

Max Activating Token Index: 680

Text #13

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2390. Min Act: -0.1700

Data Index: 6276202 (Open Web Text)

Max Activating Token Index: 795

Text #14

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.1676. Min Act: -0.1700

Data Index: 1282969 (Open Web Text)

Max Activating Token Index: 364

Text #15

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.1917. Min Act: -0.1700

Data Index: 6874336 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.1996. Min Act: -0.1700

Data Index: 613590 (Open Web Text)

Max Activating Token Index: 2

Text #17

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.1827. Min Act: -0.1700

Data Index: 7099366 (Open Web Text)

Max Activating Token Index: 1008

Text #18

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2009. Min Act: -0.1700

Data Index: 3875187 (Open Web Text)

Max Activating Token Index: 513

Text #19

Max Range: 5.3561. Min Range: -5.3561

Max Act: 5.2018. Min Act: -0.1700

Data Index: 1660676 (Open Web Text)

Max Activating Token Index: 778