Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1422 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3735. Min Act: -0.1700

Data Index: 4414049 (Open Web Text)

Max Activating Token Index: 184

Text #1

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3539. Min Act: -0.1700

Data Index: 4970978 (Open Web Text)

Max Activating Token Index: 648

Text #2

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3184. Min Act: -0.1700

Data Index: 5941763 (Open Web Text)

Max Activating Token Index: 193

Text #3

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3361. Min Act: -0.1700

Data Index: 6565092 (Open Web Text)

Max Activating Token Index: 885

Text #4

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3109. Min Act: -0.1700

Data Index: 321981 (Open Web Text)

Max Activating Token Index: 948

Text #5

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3035. Min Act: -0.1700

Data Index: 8229035 (Open Web Text)

Max Activating Token Index: 113

Text #6

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.3112. Min Act: -0.1700

Data Index: 8002007 (Open Web Text)

Max Activating Token Index: 490

Text #7

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2963. Min Act: -0.1700

Data Index: 7527576 (Open Web Text)

Max Activating Token Index: 96

Text #8

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2847. Min Act: -0.1700

Data Index: 2097460 (Open Web Text)

Max Activating Token Index: 522

Text #9

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2739. Min Act: -0.1700

Data Index: 8425988 (Open Web Text)

Max Activating Token Index: 337

Text #10

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2794. Min Act: -0.1700

Data Index: 7525973 (Open Web Text)

Max Activating Token Index: 912

Text #11

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2741. Min Act: -0.1700

Data Index: 7674654 (Open Web Text)

Max Activating Token Index: 132

Text #12

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2958. Min Act: -0.1700

Data Index: 3196143 (Open Web Text)

Max Activating Token Index: 712

Text #13

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2660. Min Act: -0.1700

Data Index: 4005670 (Open Web Text)

Max Activating Token Index: 424

Text #14

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2808. Min Act: -0.1700

Data Index: 2964515 (Open Web Text)

Max Activating Token Index: 253

Text #15

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2650. Min Act: -0.1700

Data Index: 4938966 (Open Web Text)

Max Activating Token Index: 302

Text #16

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2709. Min Act: -0.1700

Data Index: 7904108 (Open Web Text)

Max Activating Token Index: 18

Text #17

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2712. Min Act: -0.1700

Data Index: 4392633 (Open Web Text)

Max Activating Token Index: 141

Text #18

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2592. Min Act: -0.1700

Data Index: 7878022 (Open Web Text)

Max Activating Token Index: 756

Text #19

Max Range: 3.3735. Min Range: -3.3735

Max Act: 3.2383. Min Act: -0.1700

Data Index: 194035 (Open Web Text)

Max Activating Token Index: 762