Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1806 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6749. Min Act: -0.1700

Data Index: 6609516 (Open Web Text)

Max Activating Token Index: 438

Text #1

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6451. Min Act: -0.1700

Data Index: 7198836 (Open Web Text)

Max Activating Token Index: 428

Text #2

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6497. Min Act: -0.1700

Data Index: 55719 (Open Web Text)

Max Activating Token Index: 428

Text #3

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6525. Min Act: -0.1700

Data Index: 405841 (Open Web Text)

Max Activating Token Index: 385

Text #4

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6447. Min Act: -0.1700

Data Index: 2584543 (Open Web Text)

Max Activating Token Index: 433

Text #5

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6425. Min Act: -0.1700

Data Index: 6461181 (Open Web Text)

Max Activating Token Index: 426

Text #6

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6399. Min Act: -0.1700

Data Index: 550205 (Open Web Text)

Max Activating Token Index: 399

Text #7

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6345. Min Act: -0.1700

Data Index: 4305066 (Open Web Text)

Max Activating Token Index: 434

Text #8

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6396. Min Act: -0.1700

Data Index: 4777619 (Open Web Text)

Max Activating Token Index: 430

Text #9

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6299. Min Act: -0.1700

Data Index: 24120 (Open Web Text)

Max Activating Token Index: 461

Text #10

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6340. Min Act: -0.1700

Data Index: 2202615 (Open Web Text)

Max Activating Token Index: 414

Text #11

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6399. Min Act: -0.1700

Data Index: 3564275 (Open Web Text)

Max Activating Token Index: 456

Text #12

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6291. Min Act: -0.1700

Data Index: 409895 (Open Web Text)

Max Activating Token Index: 441

Text #13

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6327. Min Act: -0.1700

Data Index: 6131005 (Open Web Text)

Max Activating Token Index: 408

Text #14

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6383. Min Act: -0.1700

Data Index: 3078447 (Open Web Text)

Max Activating Token Index: 420

Text #15

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6256. Min Act: -0.1700

Data Index: 3181138 (Open Web Text)

Max Activating Token Index: 448

Text #16

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6327. Min Act: -0.1700

Data Index: 3404615 (Open Web Text)

Max Activating Token Index: 414

Text #17

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6255. Min Act: -0.1700

Data Index: 5645745 (Open Web Text)

Max Activating Token Index: 435

Text #18

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6191. Min Act: -0.1699

Data Index: 2353197 (Open Web Text)

Max Activating Token Index: 438

Text #19

Max Range: 0.6749. Min Range: -0.6749

Max Act: 0.6276. Min Act: -0.1700

Data Index: 2244636 (Open Web Text)

Max Activating Token Index: 445