Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1022 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0440. Min Range: -3.0440

Max Act: 3.0440. Min Act: -0.1700

Data Index: 456613 (Open Web Text)

Max Activating Token Index: 1013

Text #1

Max Range: 3.0440. Min Range: -3.0440

Max Act: 3.0224. Min Act: -0.1700

Data Index: 2254352 (Open Web Text)

Max Activating Token Index: 473

Text #2

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9857. Min Act: -0.1700

Data Index: 4516669 (Open Web Text)

Max Activating Token Index: 688

Text #3

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9740. Min Act: -0.1700

Data Index: 343631 (Open Web Text)

Max Activating Token Index: 285

Text #4

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9330. Min Act: -0.1700

Data Index: 2229006 (Open Web Text)

Max Activating Token Index: 466

Text #5

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9739. Min Act: -0.1700

Data Index: 6568723 (Open Web Text)

Max Activating Token Index: 467

Text #6

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9462. Min Act: -0.1700

Data Index: 8713728 (Open Web Text)

Max Activating Token Index: 890

Text #7

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9397. Min Act: -0.1700

Data Index: 746015 (Open Web Text)

Max Activating Token Index: 423

Text #8

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9244. Min Act: -0.1700

Data Index: 3368899 (Open Web Text)

Max Activating Token Index: 376

Text #9

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9253. Min Act: -0.1700

Data Index: 97751 (Open Web Text)

Max Activating Token Index: 659

Text #10

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9187. Min Act: -0.1700

Data Index: 4022942 (Open Web Text)

Max Activating Token Index: 77

Text #11

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9086. Min Act: -0.1700

Data Index: 3031897 (Open Web Text)

Max Activating Token Index: 867

Text #12

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9020. Min Act: -0.1700

Data Index: 7747813 (Open Web Text)

Max Activating Token Index: 996

Text #13

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9119. Min Act: -0.1700

Data Index: 1933069 (Open Web Text)

Max Activating Token Index: 428

Text #14

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9100. Min Act: -0.1700

Data Index: 429151 (Open Web Text)

Max Activating Token Index: 816

Text #15

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.8934. Min Act: -0.1700

Data Index: 6110918 (Open Web Text)

Max Activating Token Index: 278

Text #16

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.9062. Min Act: -0.1700

Data Index: 7075063 (Open Web Text)

Max Activating Token Index: 590

Text #17

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.8843. Min Act: -0.1700

Data Index: 3959771 (Open Web Text)

Max Activating Token Index: 305

Text #18

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.8879. Min Act: -0.1700

Data Index: 8780832 (Open Web Text)

Max Activating Token Index: 313

Text #19

Max Range: 3.0440. Min Range: -3.0440

Max Act: 2.8925. Min Act: -0.1700

Data Index: 3791175 (Open Web Text)

Max Activating Token Index: 209