Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 873 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0381. Min Range: -4.0381

Max Act: 4.0381. Min Act: -0.1700

Data Index: 5723295 (Open Web Text)

Max Activating Token Index: 182

Text #1

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9822. Min Act: -0.1700

Data Index: 2444543 (Open Web Text)

Max Activating Token Index: 916

Text #2

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9659. Min Act: -0.1700

Data Index: 8679608 (Open Web Text)

Max Activating Token Index: 231

Text #3

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9398. Min Act: -0.1700

Data Index: 5158005 (Open Web Text)

Max Activating Token Index: 656

Text #4

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9588. Min Act: -0.1700

Data Index: 3123984 (Open Web Text)

Max Activating Token Index: 539

Text #5

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9818. Min Act: -0.1700

Data Index: 5835486 (Open Web Text)

Max Activating Token Index: 693

Text #6

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9060. Min Act: -0.1700

Data Index: 4439761 (Open Web Text)

Max Activating Token Index: 835

Text #7

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9187. Min Act: -0.1700

Data Index: 5009437 (Open Web Text)

Max Activating Token Index: 144

Text #8

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9030. Min Act: -0.1700

Data Index: 8374165 (Open Web Text)

Max Activating Token Index: 803

Text #9

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8931. Min Act: -0.1700

Data Index: 289540 (Open Web Text)

Max Activating Token Index: 573

Text #10

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.9053. Min Act: -0.1700

Data Index: 3539724 (Open Web Text)

Max Activating Token Index: 586

Text #11

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8949. Min Act: -0.1700

Data Index: 2770385 (Open Web Text)

Max Activating Token Index: 138

Text #12

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8986. Min Act: -0.1700

Data Index: 7527502 (Open Web Text)

Max Activating Token Index: 580

Text #13

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8821. Min Act: -0.1700

Data Index: 4189611 (Open Web Text)

Max Activating Token Index: 152

Text #14

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8723. Min Act: -0.1700

Data Index: 5794936 (Open Web Text)

Max Activating Token Index: 434

Text #15

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8962. Min Act: -0.1700

Data Index: 749512 (Open Web Text)

Max Activating Token Index: 49

Text #16

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8821. Min Act: -0.1700

Data Index: 289554 (Open Web Text)

Max Activating Token Index: 35

Text #17

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8838. Min Act: -0.1700

Data Index: 3667291 (Open Web Text)

Max Activating Token Index: 507

Text #18

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8894. Min Act: -0.1700

Data Index: 289549 (Open Web Text)

Max Activating Token Index: 590

Text #19

Max Range: 4.0381. Min Range: -4.0381

Max Act: 3.8630. Min Act: -0.1700

Data Index: 3261919 (Open Web Text)

Max Activating Token Index: 460