Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3063 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.5212. Min Act: -0.1700

Data Index: 1018207 (Open Web Text)

Max Activating Token Index: 462

Text #1

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.5389. Min Act: -0.1700

Data Index: 1963331 (Open Web Text)

Max Activating Token Index: 477

Text #2

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.5045. Min Act: -0.1700

Data Index: 6109742 (Open Web Text)

Max Activating Token Index: 213

Text #3

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.4824. Min Act: -0.1700

Data Index: 6870425 (Open Web Text)

Max Activating Token Index: 825

Text #4

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.4070. Min Act: -0.1700

Data Index: 1281658 (Open Web Text)

Max Activating Token Index: 366

Text #5

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.4278. Min Act: -0.1700

Data Index: 7901653 (Open Web Text)

Max Activating Token Index: 745

Text #6

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3766. Min Act: -0.1700

Data Index: 1585544 (Open Web Text)

Max Activating Token Index: 304

Text #7

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3771. Min Act: -0.1700

Data Index: 1521825 (Open Web Text)

Max Activating Token Index: 251

Text #8

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3849. Min Act: -0.1700

Data Index: 1616173 (Open Web Text)

Max Activating Token Index: 787

Text #9

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3658. Min Act: -0.1700

Data Index: 844196 (Open Web Text)

Max Activating Token Index: 753

Text #10

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3233. Min Act: -0.1700

Data Index: 3627520 (Open Web Text)

Max Activating Token Index: 1001

Text #11

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3258. Min Act: -0.1700

Data Index: 1712460 (Open Web Text)

Max Activating Token Index: 918

Text #12

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3336. Min Act: -0.1700

Data Index: 955014 (Open Web Text)

Max Activating Token Index: 51

Text #13

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3064. Min Act: -0.1700

Data Index: 2195660 (Open Web Text)

Max Activating Token Index: 91

Text #14

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3108. Min Act: -0.1700

Data Index: 3296748 (Open Web Text)

Max Activating Token Index: 122

Text #15

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3094. Min Act: -0.1700

Data Index: 2141282 (Open Web Text)

Max Activating Token Index: 778

Text #16

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3162. Min Act: -0.1700

Data Index: 1996345 (Open Web Text)

Max Activating Token Index: 644

Text #17

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3003. Min Act: -0.1700

Data Index: 8559347 (Open Web Text)

Max Activating Token Index: 759

Text #18

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3023. Min Act: -0.1700

Data Index: 5564981 (Open Web Text)

Max Activating Token Index: 222

Text #19

Max Range: 3.5389. Min Range: -3.5389

Max Act: 3.3083. Min Act: -0.1700

Data Index: 6138224 (Open Web Text)

Max Activating Token Index: 535