Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 45 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.7718. Min Act: -0.1700

Data Index: 6511682 (Open Web Text)

Max Activating Token Index: 368

Text #1

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.8163. Min Act: -0.1700

Data Index: 4643238 (Open Web Text)

Max Activating Token Index: 150

Text #2

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.7686. Min Act: -0.1700

Data Index: 2691956 (Open Web Text)

Max Activating Token Index: 154

Text #3

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.7586. Min Act: -0.1700

Data Index: 2291967 (Open Web Text)

Max Activating Token Index: 509

Text #4

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.7172. Min Act: -0.1700

Data Index: 5133119 (Open Web Text)

Max Activating Token Index: 1004

Text #5

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.7078. Min Act: -0.1700

Data Index: 7534111 (Open Web Text)

Max Activating Token Index: 132

Text #6

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6777. Min Act: -0.1700

Data Index: 7670168 (Open Web Text)

Max Activating Token Index: 456

Text #7

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6748. Min Act: -0.1700

Data Index: 4459298 (Open Web Text)

Max Activating Token Index: 960

Text #8

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6732. Min Act: -0.1700

Data Index: 5369478 (Open Web Text)

Max Activating Token Index: 432

Text #9

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6851. Min Act: -0.1700

Data Index: 5156403 (Open Web Text)

Max Activating Token Index: 52

Text #10

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6899. Min Act: -0.1700

Data Index: 6410560 (Open Web Text)

Max Activating Token Index: 406

Text #11

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6817. Min Act: -0.1700

Data Index: 5353731 (Open Web Text)

Max Activating Token Index: 115

Text #12

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6314. Min Act: -0.1700

Data Index: 6641501 (Open Web Text)

Max Activating Token Index: 590

Text #13

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6714. Min Act: -0.1700

Data Index: 396295 (Open Web Text)

Max Activating Token Index: 52

Text #14

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6451. Min Act: -0.1700

Data Index: 1916492 (Open Web Text)

Max Activating Token Index: 798

Text #15

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6405. Min Act: -0.1700

Data Index: 2691939 (Open Web Text)

Max Activating Token Index: 325

Text #16

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.5703. Min Act: -0.1700

Data Index: 2557407 (Open Web Text)

Max Activating Token Index: 995

Text #17

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6069. Min Act: -0.1700

Data Index: 3571867 (Open Web Text)

Max Activating Token Index: 855

Text #18

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.6140. Min Act: -0.1700

Data Index: 6495069 (Open Web Text)

Max Activating Token Index: 448

Text #19

Max Range: 3.8163. Min Range: -3.8163

Max Act: 3.5715. Min Act: -0.1700

Data Index: 1259886 (Open Web Text)

Max Activating Token Index: 106