Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1328 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1104. Min Range: -4.1104

Max Act: 4.1104. Min Act: -0.1700

Data Index: 8239153 (Open Web Text)

Max Activating Token Index: 507

Text #1

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.9881. Min Act: -0.1700

Data Index: 2025613 (Open Web Text)

Max Activating Token Index: 613

Text #2

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.9553. Min Act: -0.1700

Data Index: 6094264 (Open Web Text)

Max Activating Token Index: 655

Text #3

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.9268. Min Act: -0.1700

Data Index: 8679750 (Open Web Text)

Max Activating Token Index: 196

Text #4

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.8889. Min Act: -0.1700

Data Index: 2513482 (Open Web Text)

Max Activating Token Index: 421

Text #5

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.9064. Min Act: -0.1700

Data Index: 2405131 (Open Web Text)

Max Activating Token Index: 916

Text #6

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7799. Min Act: -0.1700

Data Index: 4846133 (Open Web Text)

Max Activating Token Index: 964

Text #7

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7569. Min Act: -0.1700

Data Index: 892538 (Open Web Text)

Max Activating Token Index: 323

Text #8

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7037. Min Act: -0.1700

Data Index: 3472260 (Open Web Text)

Max Activating Token Index: 751

Text #9

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7515. Min Act: -0.1700

Data Index: 5992693 (Open Web Text)

Max Activating Token Index: 764

Text #10

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6892. Min Act: -0.1700

Data Index: 5268889 (Open Web Text)

Max Activating Token Index: 510

Text #11

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7214. Min Act: -0.1700

Data Index: 6859291 (Open Web Text)

Max Activating Token Index: 971

Text #12

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7108. Min Act: -0.1700

Data Index: 5868185 (Open Web Text)

Max Activating Token Index: 700

Text #13

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.7077. Min Act: -0.1700

Data Index: 5091007 (Open Web Text)

Max Activating Token Index: 900

Text #14

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6698. Min Act: -0.1700

Data Index: 6020535 (Open Web Text)

Max Activating Token Index: 914

Text #15

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6932. Min Act: -0.1700

Data Index: 4748718 (Open Web Text)

Max Activating Token Index: 359

Text #16

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6507. Min Act: -0.1700

Data Index: 1904007 (Open Web Text)

Max Activating Token Index: 656

Text #17

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6981. Min Act: -0.1700

Data Index: 3167371 (Open Web Text)

Max Activating Token Index: 39

Text #18

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6380. Min Act: -0.1700

Data Index: 2583559 (Open Web Text)

Max Activating Token Index: 368

Text #19

Max Range: 4.1104. Min Range: -4.1104

Max Act: 3.6601. Min Act: -0.1700

Data Index: 2227308 (Open Web Text)

Max Activating Token Index: 975