Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1904 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.8008. Min Act: -0.1700

Data Index: 4663753 (Open Web Text)

Max Activating Token Index: 808

Text #1

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.7220. Min Act: -0.1700

Data Index: 36255 (Open Web Text)

Max Activating Token Index: 697

Text #2

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.6714. Min Act: -0.1700

Data Index: 8068838 (Open Web Text)

Max Activating Token Index: 724

Text #3

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5957. Min Act: -0.1700

Data Index: 7554656 (Open Web Text)

Max Activating Token Index: 828

Text #4

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5933. Min Act: -0.1700

Data Index: 4951376 (Open Web Text)

Max Activating Token Index: 553

Text #5

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5938. Min Act: -0.1700

Data Index: 3753581 (Open Web Text)

Max Activating Token Index: 519

Text #6

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5515. Min Act: -0.1700

Data Index: 779606 (Open Web Text)

Max Activating Token Index: 833

Text #7

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5088. Min Act: -0.1700

Data Index: 7886579 (Open Web Text)

Max Activating Token Index: 1006

Text #8

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5346. Min Act: -0.1700

Data Index: 5792902 (Open Web Text)

Max Activating Token Index: 228

Text #9

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5283. Min Act: -0.1700

Data Index: 861093 (Open Web Text)

Max Activating Token Index: 773

Text #10

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5369. Min Act: -0.1700

Data Index: 7333686 (Open Web Text)

Max Activating Token Index: 904

Text #11

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4979. Min Act: -0.1700

Data Index: 2107953 (Open Web Text)

Max Activating Token Index: 604

Text #12

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.5078. Min Act: -0.1700

Data Index: 2698867 (Open Web Text)

Max Activating Token Index: 907

Text #13

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4830. Min Act: -0.1700

Data Index: 938796 (Open Web Text)

Max Activating Token Index: 936

Text #14

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4557. Min Act: -0.1700

Data Index: 6492095 (Open Web Text)

Max Activating Token Index: 277

Text #15

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4797. Min Act: -0.1700

Data Index: 944568 (Open Web Text)

Max Activating Token Index: 786

Text #16

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4879. Min Act: -0.1700

Data Index: 5121329 (Open Web Text)

Max Activating Token Index: 429

Text #17

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4340. Min Act: -0.1700

Data Index: 6692341 (Open Web Text)

Max Activating Token Index: 742

Text #18

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4564. Min Act: -0.1700

Data Index: 5732020 (Open Web Text)

Max Activating Token Index: 551

Text #19

Max Range: 4.8008. Min Range: -4.8008

Max Act: 4.4142. Min Act: -0.1700

Data Index: 2247213 (Open Web Text)

Max Activating Token Index: 585