Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 914 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.6032. Min Act: -0.1700

Data Index: 4528383 (Open Web Text)

Max Activating Token Index: 672

Text #1

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5749. Min Act: -0.1700

Data Index: 7526141 (Open Web Text)

Max Activating Token Index: 479

Text #2

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5353. Min Act: -0.1700

Data Index: 5954382 (Open Web Text)

Max Activating Token Index: 321

Text #3

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5550. Min Act: -0.1700

Data Index: 1768739 (Open Web Text)

Max Activating Token Index: 509

Text #4

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5324. Min Act: -0.1700

Data Index: 709537 (Open Web Text)

Max Activating Token Index: 584

Text #5

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5174. Min Act: -0.1700

Data Index: 6960328 (Open Web Text)

Max Activating Token Index: 480

Text #6

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5298. Min Act: -0.1700

Data Index: 1506446 (Open Web Text)

Max Activating Token Index: 716

Text #7

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5135. Min Act: -0.1700

Data Index: 4583085 (Open Web Text)

Max Activating Token Index: 311

Text #8

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5042. Min Act: -0.1700

Data Index: 7592545 (Open Web Text)

Max Activating Token Index: 735

Text #9

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5171. Min Act: -0.1700

Data Index: 3160649 (Open Web Text)

Max Activating Token Index: 464

Text #10

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.5037. Min Act: -0.1700

Data Index: 3251362 (Open Web Text)

Max Activating Token Index: 301

Text #11

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4959. Min Act: -0.1700

Data Index: 5652359 (Open Web Text)

Max Activating Token Index: 154

Text #12

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4839. Min Act: -0.1700

Data Index: 3474289 (Open Web Text)

Max Activating Token Index: 783

Text #13

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4763. Min Act: -0.1700

Data Index: 5878070 (Open Web Text)

Max Activating Token Index: 508

Text #14

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4828. Min Act: -0.1700

Data Index: 4910223 (Open Web Text)

Max Activating Token Index: 282

Text #15

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4647. Min Act: -0.1700

Data Index: 6824312 (Open Web Text)

Max Activating Token Index: 139

Text #16

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4469. Min Act: -0.1700

Data Index: 6966521 (Open Web Text)

Max Activating Token Index: 353

Text #17

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4652. Min Act: -0.1700

Data Index: 4682364 (Open Web Text)

Max Activating Token Index: 571

Text #18

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4627. Min Act: -0.1700

Data Index: 3588660 (Open Web Text)

Max Activating Token Index: 516

Text #19

Max Range: 3.6032. Min Range: -3.6032

Max Act: 3.4543. Min Act: -0.1700

Data Index: 4549200 (Open Web Text)

Max Activating Token Index: 303