Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2326 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.6450. Min Act: -0.1700

Data Index: 4458090 (Open Web Text)

Max Activating Token Index: 783

Text #1

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5946. Min Act: -0.1700

Data Index: 3448798 (Open Web Text)

Max Activating Token Index: 30

Text #2

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5565. Min Act: -0.1700

Data Index: 8242326 (Open Web Text)

Max Activating Token Index: 186

Text #3

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5398. Min Act: -0.1700

Data Index: 2868052 (Open Web Text)

Max Activating Token Index: 711

Text #4

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5543. Min Act: -0.1700

Data Index: 5985754 (Open Web Text)

Max Activating Token Index: 169

Text #5

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5151. Min Act: -0.1700

Data Index: 6280534 (Open Web Text)

Max Activating Token Index: 241

Text #6

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5142. Min Act: -0.1700

Data Index: 5879909 (Open Web Text)

Max Activating Token Index: 796

Text #7

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5189. Min Act: -0.1700

Data Index: 2443751 (Open Web Text)

Max Activating Token Index: 162

Text #8

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5301. Min Act: -0.1700

Data Index: 8242323 (Open Web Text)

Max Activating Token Index: 363

Text #9

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5407. Min Act: -0.1700

Data Index: 4851698 (Open Web Text)

Max Activating Token Index: 981

Text #10

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5241. Min Act: -0.1648

Data Index: 8242325 (Open Web Text)

Max Activating Token Index: 245

Text #11

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4766. Min Act: -0.1700

Data Index: 3131079 (Open Web Text)

Max Activating Token Index: 582

Text #12

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4951. Min Act: -0.1700

Data Index: 6011538 (Open Web Text)

Max Activating Token Index: 290

Text #13

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4997. Min Act: -0.1700

Data Index: 6505287 (Open Web Text)

Max Activating Token Index: 395

Text #14

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4853. Min Act: -0.1700

Data Index: 3969267 (Open Web Text)

Max Activating Token Index: 452

Text #15

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4865. Min Act: -0.1700

Data Index: 1202381 (Open Web Text)

Max Activating Token Index: 218

Text #16

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4928. Min Act: -0.1700

Data Index: 6557889 (Open Web Text)

Max Activating Token Index: 350

Text #17

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5049. Min Act: -0.1700

Data Index: 2698359 (Open Web Text)

Max Activating Token Index: 420

Text #18

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5042. Min Act: -0.1700

Data Index: 7972737 (Open Web Text)

Max Activating Token Index: 690

Text #19

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4805. Min Act: -0.1700

Data Index: 8183585 (Open Web Text)

Max Activating Token Index: 300