Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 732 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.2259. Min Act: -0.1700

Data Index: 3252114 (Open Web Text)

Max Activating Token Index: 102

Text #1

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.1846. Min Act: -0.1700

Data Index: 8657802 (Open Web Text)

Max Activating Token Index: 938

Text #2

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.1603. Min Act: -0.1700

Data Index: 7083673 (Open Web Text)

Max Activating Token Index: 1004

Text #3

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.1168. Min Act: -0.1700

Data Index: 5521084 (Open Web Text)

Max Activating Token Index: 543

Text #4

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0878. Min Act: -0.1700

Data Index: 7034771 (Open Web Text)

Max Activating Token Index: 378

Text #5

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0956. Min Act: -0.1700

Data Index: 7036220 (Open Web Text)

Max Activating Token Index: 918

Text #6

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0625. Min Act: -0.1700

Data Index: 4873021 (Open Web Text)

Max Activating Token Index: 589

Text #7

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0547. Min Act: -0.1700

Data Index: 8281838 (Open Web Text)

Max Activating Token Index: 392

Text #8

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0400. Min Act: -0.1700

Data Index: 268907 (Open Web Text)

Max Activating Token Index: 863

Text #9

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0407. Min Act: -0.1700

Data Index: 2365499 (Open Web Text)

Max Activating Token Index: 642

Text #10

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0306. Min Act: -0.1700

Data Index: 4888098 (Open Web Text)

Max Activating Token Index: 129

Text #11

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0217. Min Act: -0.1700

Data Index: 230850 (Open Web Text)

Max Activating Token Index: 451

Text #12

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9768. Min Act: -0.1700

Data Index: 746282 (Open Web Text)

Max Activating Token Index: 411

Text #13

Max Range: 3.2259. Min Range: -3.2259

Max Act: 3.0110. Min Act: -0.1700

Data Index: 1692509 (Open Web Text)

Max Activating Token Index: 393

Text #14

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9804. Min Act: -0.1700

Data Index: 489224 (Open Web Text)

Max Activating Token Index: 497

Text #15

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9857. Min Act: -0.1700

Data Index: 2507800 (Open Web Text)

Max Activating Token Index: 200

Text #16

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9750. Min Act: -0.1700

Data Index: 7892613 (Open Web Text)

Max Activating Token Index: 407

Text #17

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9625. Min Act: -0.1700

Data Index: 5744302 (Open Web Text)

Max Activating Token Index: 72

Text #18

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9809. Min Act: -0.1700

Data Index: 5179124 (Open Web Text)

Max Activating Token Index: 320

Text #19

Max Range: 3.2259. Min Range: -3.2259

Max Act: 2.9451. Min Act: -0.1700

Data Index: 8479522 (Open Web Text)

Max Activating Token Index: 740