Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2870 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.9156. Min Act: -0.1700

Data Index: 7164480 (Open Web Text)

Max Activating Token Index: 879

Text #1

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8465. Min Act: -0.1700

Data Index: 5434610 (Open Web Text)

Max Activating Token Index: 449

Text #2

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8769. Min Act: -0.1700

Data Index: 8687689 (Open Web Text)

Max Activating Token Index: 212

Text #3

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8658. Min Act: -0.1700

Data Index: 8060489 (Open Web Text)

Max Activating Token Index: 716

Text #4

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8826. Min Act: -0.1700

Data Index: 1580571 (Open Web Text)

Max Activating Token Index: 1011

Text #5

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8198. Min Act: -0.1700

Data Index: 5594310 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8319. Min Act: -0.1700

Data Index: 8034881 (Open Web Text)

Max Activating Token Index: 953

Text #7

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7941. Min Act: -0.1700

Data Index: 3033790 (Open Web Text)

Max Activating Token Index: 964

Text #8

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7825. Min Act: -0.1700

Data Index: 4948806 (Open Web Text)

Max Activating Token Index: 380

Text #9

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7943. Min Act: -0.1700

Data Index: 3700306 (Open Web Text)

Max Activating Token Index: 484

Text #10

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.8179. Min Act: -0.1700

Data Index: 365916 (Open Web Text)

Max Activating Token Index: 137

Text #11

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7867. Min Act: -0.1700

Data Index: 5408465 (Open Web Text)

Max Activating Token Index: 329

Text #12

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7405. Min Act: -0.1700

Data Index: 6828606 (Open Web Text)

Max Activating Token Index: 365

Text #13

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7484. Min Act: -0.1700

Data Index: 8210211 (Open Web Text)

Max Activating Token Index: 467

Text #14

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7595. Min Act: -0.1700

Data Index: 74610 (Open Web Text)

Max Activating Token Index: 332

Text #15

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7343. Min Act: -0.1700

Data Index: 1263911 (Open Web Text)

Max Activating Token Index: 892

Text #16

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7449. Min Act: -0.1700

Data Index: 5635723 (Open Web Text)

Max Activating Token Index: 491

Text #17

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7323. Min Act: -0.1700

Data Index: 2846928 (Open Web Text)

Max Activating Token Index: 412

Text #18

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7312. Min Act: -0.1700

Data Index: 2718303 (Open Web Text)

Max Activating Token Index: 427

Text #19

Max Range: 3.9156. Min Range: -3.9156

Max Act: 3.7264. Min Act: -0.1700

Data Index: 1426671 (Open Web Text)

Max Activating Token Index: 901