Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 933 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0406. Min Range: -6.0406

Max Act: 6.0406. Min Act: -0.1700

Data Index: 6808371 (Open Web Text)

Max Activating Token Index: 665

Text #1

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.8539. Min Act: -0.1700

Data Index: 3912434 (Open Web Text)

Max Activating Token Index: 681

Text #2

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.8644. Min Act: -0.1700

Data Index: 2052567 (Open Web Text)

Max Activating Token Index: 499

Text #3

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.8228. Min Act: -0.1700

Data Index: 7804798 (Open Web Text)

Max Activating Token Index: 537

Text #4

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.8256. Min Act: -0.1700

Data Index: 6572736 (Open Web Text)

Max Activating Token Index: 753

Text #5

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.7829. Min Act: -0.1700

Data Index: 4756604 (Open Web Text)

Max Activating Token Index: 627

Text #6

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.7505. Min Act: -0.1700

Data Index: 5492957 (Open Web Text)

Max Activating Token Index: 1014

Text #7

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.7533. Min Act: -0.1700

Data Index: 7910316 (Open Web Text)

Max Activating Token Index: 912

Text #8

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6702. Min Act: -0.1700

Data Index: 2614628 (Open Web Text)

Max Activating Token Index: 166

Text #9

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.7008. Min Act: -0.1700

Data Index: 306104 (Open Web Text)

Max Activating Token Index: 213

Text #10

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6908. Min Act: -0.1700

Data Index: 5075844 (Open Web Text)

Max Activating Token Index: 604

Text #11

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6823. Min Act: -0.1700

Data Index: 4823347 (Open Web Text)

Max Activating Token Index: 439

Text #12

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6507. Min Act: -0.1700

Data Index: 2169873 (Open Web Text)

Max Activating Token Index: 740

Text #13

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6505. Min Act: -0.1700

Data Index: 6427246 (Open Web Text)

Max Activating Token Index: 575

Text #14

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6428. Min Act: -0.1700

Data Index: 6308518 (Open Web Text)

Max Activating Token Index: 936

Text #15

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6654. Min Act: -0.1700

Data Index: 3431832 (Open Web Text)

Max Activating Token Index: 231

Text #16

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6314. Min Act: -0.1700

Data Index: 5031275 (Open Web Text)

Max Activating Token Index: 187

Text #17

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6156. Min Act: -0.1700

Data Index: 1611312 (Open Web Text)

Max Activating Token Index: 906

Text #18

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6114. Min Act: -0.1700

Data Index: 1128905 (Open Web Text)

Max Activating Token Index: 727

Text #19

Max Range: 6.0406. Min Range: -6.0406

Max Act: 5.6135. Min Act: -0.1700

Data Index: 7524681 (Open Web Text)

Max Activating Token Index: 236