Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1027 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.8175. Min Act: -0.1700

Data Index: 5760665 (Open Web Text)

Max Activating Token Index: 875

Text #1

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.5955. Min Act: -0.1700

Data Index: 7180206 (Open Web Text)

Max Activating Token Index: 834

Text #2

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4915. Min Act: -0.1700

Data Index: 1163158 (Open Web Text)

Max Activating Token Index: 717

Text #3

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.5161. Min Act: -0.1700

Data Index: 1344245 (Open Web Text)

Max Activating Token Index: 737

Text #4

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.5206. Min Act: -0.1700

Data Index: 5398081 (Open Web Text)

Max Activating Token Index: 1002

Text #5

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4684. Min Act: -0.1700

Data Index: 8026338 (Open Web Text)

Max Activating Token Index: 908

Text #6

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4113. Min Act: -0.1700

Data Index: 2629646 (Open Web Text)

Max Activating Token Index: 751

Text #7

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4121. Min Act: -0.1700

Data Index: 441481 (Open Web Text)

Max Activating Token Index: 793

Text #8

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4217. Min Act: -0.1700

Data Index: 5370012 (Open Web Text)

Max Activating Token Index: 1012

Text #9

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4166. Min Act: -0.1700

Data Index: 1211163 (Open Web Text)

Max Activating Token Index: 998

Text #10

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.4000. Min Act: -0.1700

Data Index: 2617342 (Open Web Text)

Max Activating Token Index: 256

Text #11

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.3784. Min Act: -0.1700

Data Index: 2455622 (Open Web Text)

Max Activating Token Index: 594

Text #12

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.3605. Min Act: -0.1700

Data Index: 2719403 (Open Web Text)

Max Activating Token Index: 463

Text #13

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.3345. Min Act: -0.1700

Data Index: 135882 (Open Web Text)

Max Activating Token Index: 311

Text #14

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.3522. Min Act: -0.1700

Data Index: 5134839 (Open Web Text)

Max Activating Token Index: 86

Text #15

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.2848. Min Act: -0.1700

Data Index: 6956433 (Open Web Text)

Max Activating Token Index: 788

Text #16

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.2869. Min Act: -0.1700

Data Index: 7562807 (Open Web Text)

Max Activating Token Index: 852

Text #17

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.2781. Min Act: -0.1700

Data Index: 6096024 (Open Web Text)

Max Activating Token Index: 624

Text #18

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.2788. Min Act: -0.1700

Data Index: 3352510 (Open Web Text)

Max Activating Token Index: 501

Text #19

Max Range: 4.8175. Min Range: -4.8175

Max Act: 4.3021. Min Act: -0.1700

Data Index: 4861631 (Open Web Text)

Max Activating Token Index: 679