Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1669 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0872. Min Act: -0.1700

Data Index: 49787 (Open Web Text)

Max Activating Token Index: 414

Text #1

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0416. Min Act: -0.1700

Data Index: 2618855 (Open Web Text)

Max Activating Token Index: 247

Text #2

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0550. Min Act: -0.1700

Data Index: 6831148 (Open Web Text)

Max Activating Token Index: 498

Text #3

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0350. Min Act: -0.1700

Data Index: 5044063 (Open Web Text)

Max Activating Token Index: 52

Text #4

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0120. Min Act: -0.1700

Data Index: 262852 (Open Web Text)

Max Activating Token Index: 866

Text #5

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0447. Min Act: -0.1700

Data Index: 5844276 (Open Web Text)

Max Activating Token Index: 317

Text #6

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0143. Min Act: -0.1700

Data Index: 5044080 (Open Web Text)

Max Activating Token Index: 25

Text #7

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0100. Min Act: -0.1700

Data Index: 4214825 (Open Web Text)

Max Activating Token Index: 424

Text #8

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0035. Min Act: -0.1700

Data Index: 4214827 (Open Web Text)

Max Activating Token Index: 98

Text #9

Max Range: 3.0872. Min Range: -3.0872

Max Act: 3.0032. Min Act: -0.1700

Data Index: 3399878 (Open Web Text)

Max Activating Token Index: 636

Text #10

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9688. Min Act: -0.1700

Data Index: 4782785 (Open Web Text)

Max Activating Token Index: 392

Text #11

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9839. Min Act: -0.1700

Data Index: 732610 (Open Web Text)

Max Activating Token Index: 72

Text #12

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9734. Min Act: -0.1700

Data Index: 6037165 (Open Web Text)

Max Activating Token Index: 734

Text #13

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9653. Min Act: -0.1700

Data Index: 5916012 (Open Web Text)

Max Activating Token Index: 107

Text #14

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9779. Min Act: -0.1700

Data Index: 6572553 (Open Web Text)

Max Activating Token Index: 73

Text #15

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9453. Min Act: -0.1700

Data Index: 6634584 (Open Web Text)

Max Activating Token Index: 408

Text #16

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9596. Min Act: -0.1700

Data Index: 175152 (Open Web Text)

Max Activating Token Index: 369

Text #17

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9579. Min Act: -0.1700

Data Index: 7222251 (Open Web Text)

Max Activating Token Index: 32

Text #18

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9675. Min Act: -0.1700

Data Index: 2737408 (Open Web Text)

Max Activating Token Index: 612

Text #19

Max Range: 3.0872. Min Range: -3.0872

Max Act: 2.9544. Min Act: -0.1700

Data Index: 3316304 (Open Web Text)

Max Activating Token Index: 440