Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2564 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.9447. Min Act: -0.1700

Data Index: 1169261 (Open Web Text)

Max Activating Token Index: 78

Text #1

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.8470. Min Act: -0.1700

Data Index: 8411998 (Open Web Text)

Max Activating Token Index: 443

Text #2

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7974. Min Act: -0.1700

Data Index: 4248294 (Open Web Text)

Max Activating Token Index: 169

Text #3

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7890. Min Act: -0.1700

Data Index: 2238442 (Open Web Text)

Max Activating Token Index: 322

Text #4

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7252. Min Act: -0.1700

Data Index: 3650106 (Open Web Text)

Max Activating Token Index: 616

Text #5

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7664. Min Act: -0.1700

Data Index: 6007311 (Open Web Text)

Max Activating Token Index: 629

Text #6

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6896. Min Act: -0.1700

Data Index: 7092283 (Open Web Text)

Max Activating Token Index: 921

Text #7

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7612. Min Act: -0.1700

Data Index: 341411 (Open Web Text)

Max Activating Token Index: 344

Text #8

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7545. Min Act: -0.1700

Data Index: 5820485 (Open Web Text)

Max Activating Token Index: 95

Text #9

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7440. Min Act: -0.1700

Data Index: 885964 (Open Web Text)

Max Activating Token Index: 914

Text #10

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7066. Min Act: -0.1700

Data Index: 685322 (Open Web Text)

Max Activating Token Index: 700

Text #11

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6656. Min Act: -0.1700

Data Index: 2950179 (Open Web Text)

Max Activating Token Index: 492

Text #12

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6941. Min Act: -0.1700

Data Index: 5806775 (Open Web Text)

Max Activating Token Index: 909

Text #13

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7397. Min Act: -0.1700

Data Index: 2527275 (Open Web Text)

Max Activating Token Index: 415

Text #14

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.7198. Min Act: -0.1700

Data Index: 1097220 (Open Web Text)

Max Activating Token Index: 687

Text #15

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6997. Min Act: -0.1700

Data Index: 356391 (Open Web Text)

Max Activating Token Index: 914

Text #16

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6490. Min Act: -0.1700

Data Index: 3713796 (Open Web Text)

Max Activating Token Index: 598

Text #17

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6273. Min Act: -0.1700

Data Index: 5464727 (Open Web Text)

Max Activating Token Index: 941

Text #18

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6872. Min Act: -0.1700

Data Index: 2856280 (Open Web Text)

Max Activating Token Index: 986

Text #19

Max Range: 3.9447. Min Range: -3.9447

Max Act: 3.6631. Min Act: -0.1700

Data Index: 1351107 (Open Web Text)

Max Activating Token Index: 569