Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1124 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.7146. Min Act: -0.1700

Data Index: 6307352 (Open Web Text)

Max Activating Token Index: 295

Text #1

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.7524. Min Act: -0.1700

Data Index: 802042 (Open Web Text)

Max Activating Token Index: 618

Text #2

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.7156. Min Act: -0.1700

Data Index: 2476955 (Open Web Text)

Max Activating Token Index: 912

Text #3

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.6759. Min Act: -0.1700

Data Index: 3991437 (Open Web Text)

Max Activating Token Index: 823

Text #4

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.6545. Min Act: -0.1700

Data Index: 5360837 (Open Web Text)

Max Activating Token Index: 334

Text #5

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.6237. Min Act: -0.1700

Data Index: 4815573 (Open Web Text)

Max Activating Token Index: 212

Text #6

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.6213. Min Act: -0.1700

Data Index: 2848743 (Open Web Text)

Max Activating Token Index: 581

Text #7

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5451. Min Act: -0.1700

Data Index: 1826417 (Open Web Text)

Max Activating Token Index: 365

Text #8

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5582. Min Act: -0.1700

Data Index: 6964698 (Open Web Text)

Max Activating Token Index: 618

Text #9

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5807. Min Act: -0.1700

Data Index: 2659992 (Open Web Text)

Max Activating Token Index: 904

Text #10

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5151. Min Act: -0.1700

Data Index: 3222459 (Open Web Text)

Max Activating Token Index: 863

Text #11

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5234. Min Act: -0.1700

Data Index: 2043470 (Open Web Text)

Max Activating Token Index: 989

Text #12

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5393. Min Act: -0.1700

Data Index: 5312331 (Open Web Text)

Max Activating Token Index: 252

Text #13

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5125. Min Act: -0.1700

Data Index: 1153884 (Open Web Text)

Max Activating Token Index: 973

Text #14

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.4635. Min Act: -0.1700

Data Index: 1413916 (Open Web Text)

Max Activating Token Index: 781

Text #15

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5301. Min Act: -0.1700

Data Index: 5529234 (Open Web Text)

Max Activating Token Index: 397

Text #16

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.5225. Min Act: -0.1700

Data Index: 6241480 (Open Web Text)

Max Activating Token Index: 820

Text #17

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.4614. Min Act: -0.1700

Data Index: 1725669 (Open Web Text)

Max Activating Token Index: 205

Text #18

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.4674. Min Act: -0.1700

Data Index: 2043471 (Open Web Text)

Max Activating Token Index: 286

Text #19

Max Range: 3.7524. Min Range: -3.7524

Max Act: 3.4691. Min Act: -0.1700

Data Index: 2672328 (Open Web Text)

Max Activating Token Index: 210