Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 63 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.7547. Min Act: -0.1700

Data Index: 3775079 (Open Web Text)

Max Activating Token Index: 419

Text #1

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.7046. Min Act: -0.1700

Data Index: 4328316 (Open Web Text)

Max Activating Token Index: 975

Text #2

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.6860. Min Act: -0.1700

Data Index: 1710308 (Open Web Text)

Max Activating Token Index: 223

Text #3

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.6819. Min Act: -0.1700

Data Index: 2611541 (Open Web Text)

Max Activating Token Index: 593

Text #4

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.6425. Min Act: -0.1700

Data Index: 244910 (Open Web Text)

Max Activating Token Index: 394

Text #5

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.6307. Min Act: -0.1700

Data Index: 8253796 (Open Web Text)

Max Activating Token Index: 629

Text #6

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.6101. Min Act: -0.1700

Data Index: 1399303 (Open Web Text)

Max Activating Token Index: 561

Text #7

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.6054. Min Act: -0.1700

Data Index: 7136489 (Open Web Text)

Max Activating Token Index: 527

Text #8

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5845. Min Act: -0.1700

Data Index: 3598423 (Open Web Text)

Max Activating Token Index: 43

Text #9

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5728. Min Act: -0.1700

Data Index: 2611536 (Open Web Text)

Max Activating Token Index: 460

Text #10

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5381. Min Act: -0.1700

Data Index: 754826 (Open Web Text)

Max Activating Token Index: 897

Text #11

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5726. Min Act: -0.1700

Data Index: 2119356 (Open Web Text)

Max Activating Token Index: 112

Text #12

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5700. Min Act: -0.1700

Data Index: 6283640 (Open Web Text)

Max Activating Token Index: 492

Text #13

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5671. Min Act: -0.1700

Data Index: 6776933 (Open Web Text)

Max Activating Token Index: 130

Text #14

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5246. Min Act: -0.1700

Data Index: 1785568 (Open Web Text)

Max Activating Token Index: 86

Text #15

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5473. Min Act: -0.1700

Data Index: 1453733 (Open Web Text)

Max Activating Token Index: 125

Text #16

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.4942. Min Act: -0.1700

Data Index: 8519325 (Open Web Text)

Max Activating Token Index: 595

Text #17

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5163. Min Act: -0.1700

Data Index: 2584591 (Open Web Text)

Max Activating Token Index: 601

Text #18

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.5051. Min Act: -0.1700

Data Index: 8195135 (Open Web Text)

Max Activating Token Index: 371

Text #19

Max Range: 3.7547. Min Range: -3.7547

Max Act: 3.4991. Min Act: -0.1700

Data Index: 3183048 (Open Web Text)

Max Activating Token Index: 902