Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2798 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.4119. Min Act: -0.1700

Data Index: 6482567 (Open Web Text)

Max Activating Token Index: 829

Text #1

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.4824. Min Act: -0.1700

Data Index: 611493 (Open Web Text)

Max Activating Token Index: 403

Text #2

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.3461. Min Act: -0.1700

Data Index: 317033 (Open Web Text)

Max Activating Token Index: 294

Text #3

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.3338. Min Act: -0.1699

Data Index: 1737147 (Open Web Text)

Max Activating Token Index: 312

Text #4

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.3334. Min Act: -0.1700

Data Index: 6951167 (Open Web Text)

Max Activating Token Index: 485

Text #5

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.3069. Min Act: -0.1700

Data Index: 7294440 (Open Web Text)

Max Activating Token Index: 234

Text #6

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.3994. Min Act: -0.1700

Data Index: 5199591 (Open Web Text)

Max Activating Token Index: 856

Text #7

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.3669. Min Act: -0.1700

Data Index: 6251630 (Open Web Text)

Max Activating Token Index: 900

Text #8

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2737. Min Act: -0.1700

Data Index: 5919346 (Open Web Text)

Max Activating Token Index: 753

Text #9

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2488. Min Act: -0.1700

Data Index: 4281370 (Open Web Text)

Max Activating Token Index: 913

Text #10

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2609. Min Act: -0.1700

Data Index: 3506491 (Open Web Text)

Max Activating Token Index: 328

Text #11

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2169. Min Act: -0.1700

Data Index: 7385662 (Open Web Text)

Max Activating Token Index: 964

Text #12

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2638. Min Act: -0.1700

Data Index: 1997930 (Open Web Text)

Max Activating Token Index: 417

Text #13

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2582. Min Act: -0.1700

Data Index: 3803470 (Open Web Text)

Max Activating Token Index: 325

Text #14

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.1693. Min Act: -0.1700

Data Index: 2179697 (Open Web Text)

Max Activating Token Index: 367

Text #15

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2913. Min Act: -0.1700

Data Index: 4768877 (Open Web Text)

Max Activating Token Index: 758

Text #16

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.1546. Min Act: -0.1700

Data Index: 2283154 (Open Web Text)

Max Activating Token Index: 327

Text #17

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2498. Min Act: -0.1700

Data Index: 6658319 (Open Web Text)

Max Activating Token Index: 405

Text #18

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2062. Min Act: -0.1700

Data Index: 545595 (Open Web Text)

Max Activating Token Index: 886

Text #19

Max Range: 5.4824. Min Range: -5.4824

Max Act: 5.2470. Min Act: -0.1700

Data Index: 4132129 (Open Web Text)

Max Activating Token Index: 396