Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 202 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.3384. Min Act: -0.1700

Data Index: 5287594 (Open Web Text)

Max Activating Token Index: 327

Text #1

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.3103. Min Act: -0.1700

Data Index: 3136400 (Open Web Text)

Max Activating Token Index: 222

Text #2

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.3030. Min Act: -0.1700

Data Index: 3943037 (Open Web Text)

Max Activating Token Index: 896

Text #3

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.3115. Min Act: -0.1700

Data Index: 2457419 (Open Web Text)

Max Activating Token Index: 757

Text #4

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2813. Min Act: -0.1700

Data Index: 887711 (Open Web Text)

Max Activating Token Index: 356

Text #5

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2830. Min Act: -0.1700

Data Index: 2269093 (Open Web Text)

Max Activating Token Index: 824

Text #6

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.3002. Min Act: -0.1700

Data Index: 2119542 (Open Web Text)

Max Activating Token Index: 978

Text #7

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2706. Min Act: -0.1700

Data Index: 8749575 (Open Web Text)

Max Activating Token Index: 523

Text #8

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2637. Min Act: -0.1700

Data Index: 5929401 (Open Web Text)

Max Activating Token Index: 906

Text #9

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2638. Min Act: -0.1700

Data Index: 5808180 (Open Web Text)

Max Activating Token Index: 545

Text #10

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2634. Min Act: -0.1700

Data Index: 4714317 (Open Web Text)

Max Activating Token Index: 452

Text #11

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2672. Min Act: -0.1700

Data Index: 6495058 (Open Web Text)

Max Activating Token Index: 101

Text #12

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2724. Min Act: -0.1700

Data Index: 1242858 (Open Web Text)

Max Activating Token Index: 257

Text #13

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2624. Min Act: -0.1700

Data Index: 8606056 (Open Web Text)

Max Activating Token Index: 663

Text #14

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2512. Min Act: -0.1700

Data Index: 8071620 (Open Web Text)

Max Activating Token Index: 381

Text #15

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2214. Min Act: -0.1700

Data Index: 3571818 (Open Web Text)

Max Activating Token Index: 197

Text #16

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2265. Min Act: -0.1700

Data Index: 3995573 (Open Web Text)

Max Activating Token Index: 985

Text #17

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2301. Min Act: -0.1700

Data Index: 6475841 (Open Web Text)

Max Activating Token Index: 1006

Text #18

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2520. Min Act: -0.1700

Data Index: 5199803 (Open Web Text)

Max Activating Token Index: 1006

Text #19

Max Range: 3.3384. Min Range: -3.3384

Max Act: 3.2384. Min Act: -0.1700

Data Index: 6607943 (Open Web Text)

Max Activating Token Index: 110